Listening to Students: Innovative Responses
Betsy WilsonUniversity of Washington
July 10, 20087th International JISC/CNI Conference
The Role of Library Assessment
The library community needs to invest more in data collection and analysis and to take its examples from commercial leaders that have a much more detailed and insightful understanding of their customer base and preferences. In particular, there is a need for ongoing longitudinal data and intelligence functions to provide a vital early radar warning of oncoming change.
No library we are aware of has a department devoted to the evaluation of the user, how can that be?
(Information Behaviour of the Researcher of the Future, CIBER, January 2008)
North American University Libraries with Notable Assessment Programs
• Alberta “Ongoing student surveys”
• Cornell “In-house research capability”
• Pennsylvania “Data Farm”
• Rochester “Ethnographic approaches”
• Virginia “Balanced Scorecard”
• Washington “User needs”
Building a Community of Practice:Library Assessment Conference
Co-sponsored by ARL, Virginia and Washington– 2006 Charlottesville,
Virginia (220 registrants)– 2008 Seattle, Washington
(August 4-7, 380 registrants)
– 2010 Washington D.C. area
• Focused on practitioners – Papers, panels, posters,
informal contact, workshops
ARL Sponsored Assessment
• Tools– ARL Statistics– LibQUAL+®– MINES for Libraries
• Building a Community of Practice– Library Assessment
Conferences– Service Quality Evaluation
Academy– Library Assessment blog– Workshops
• Individual Library Consultation– Dynamic Duo: Jim Self (Virginia)
and Steve Hiller (Washington– Making Library Assessment
Work (24 libraries)– Effective, Sustainable, Practical
Library Assessment (6 libraries)
UW Libraries Assessment Priorities
• Understanding how faculty and students work
• Information seeking behavior and use
• Patterns of library use• Value of library• User needs• Library contribution to
customer success• User satisfaction with
services, collections, overall
University of Washington
• Established in 1861
• Campuses in Seattle, Tacoma, and Bothell
• Public research university
Students and Learning
• Diverse students• 30,000
undergraduates• 9,300 graduate
students• 1,700 professional
students
Research and Discovery
• Collaborative and interdisciplinary
• $1 billion US in grants and contracts for research
• Engineering, technology, forestry, aerospace, marine sciences, health, biosciences, computer sciences
Seattle and Innovation
• Microsoft• Amazon• RealNetworks• Serials Solutions• Starbucks• Boeing• Biotechnology firms• Gates Foundation
University of Washington Libraries
• Large, public research library
• Over 7,000,000 volumes• 50,000 serial titles• Digital collections• Excellent service• Expert staff• Innovation• Global collections• Assessment and planning
since 1992
Culture of Assessment• …is an environment in
which decisions are based on facts, research and analysis, and where services are planned and delivered in ways which maximize positive outcomes and impacts for library clients. A culture of assessment is an integral part of the process of change and the creation of the 21st century research.
Methodological Diversity
• Qualitative and quantitative methods
• Surveys (since 1992)• Focus groups• Observation and
interviews• Usability• LibQual• Data mining• And more…
We learned that…
• Remote use preferred• Self-reliance and
unmediated• Online equals
productivity• Desktop delivery is
top priority• Undergraduates value
place
Biosciences at the UW
• Internationally ranked graduate and research programs
• Largest segment of University research community
• Receive 80% of externally funded research monies
Key Questions
• Who are the bioscience students and researchers?
• What do they do?
• Where are they physically, administratively, and intellectually?
• How do they find and use information?
• What are the barriers?
• How can the Libraries help?
• How will we know if we have made any difference?
Bioscience Findings
• Print is dead, really dead• e-journal provider• Faculty don’t come to the
physical library• Students come to the
physical library• Databases underused• Personal information
management• Buy from Amazon
More Findings
• Transaction cost from discovery to delivery
• Fragmented systems and processes
• Multi-disciplinary collaborators
• Everywhere, in scattered locations
• Independent and self-sufficient researchers
Biosciences Recommendations
• Integrate search/discovery tools into users workflow• Expand/improve information/service delivery options• Make physical libraries more inviting and easier to use
– Consolidate libraries, collections and service points– Reduce print holdings– Focus on services
• Use an integrated approach to collection allocations• Get librarians to work outside library space• Lead/partner in scholarly communications and E-science • Provide more targeted communication and marketing
Biosciences Actions 2007-08
• Appointed a Director, Cyberinfrastructure Initiatives, Biosciences, and eScience
• Strategic priorities:– Improve discovery to delivery
(WorldCat Local, etc.)– Reshape physical facilities as
discovery and learning centers – More rapid delivery services– Enhance the Libraries support
for UW’s scientific research infrastructure (DataNet, etc.)
– Do market research
Can We Generalize?Did themes raised in the interviews/focus groups
reflect the bioscience population? The campus community?
The 2007 Triennial Survey as a corroborating source:• Mode of access• Resource type importance• Sources consulted for research• Primary reasons for using Libraries web sites• Libraries contribution to work and academic
success
• Useful library services (new and/or expanded)
Reasons for In-Person Library Visits 2001
Faculty and Undergrads Visiting Weekly or More Often
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Use Collections Use space orservices
Only Collections Only space orservices
Faculty
Undergrads
Library as Place Change In Frequency of In-Person Visits 1998-2007 (weekly+)
Faculty
Faculty
Grad
Grad
Undergrad
Undergrad
20%
30%
40%
50%
60%
70%
80%
1998 2001 2004 2007
Undergrad Activities DuringLibrary Visits
(% using at least weekly in 2004/2007)
0%
10%
20%
30%
40%
50%
60%
70%
Getassistance
Look formaterial
Use libcomputers
IndividualWork
Group Work
2004 2007
Off-Campus Remote Use 1998-2007(Percentage using library services/collections at least 2x week)
Faculty
Undergrad
Grad
0%
10%
20%
30%
40%
50%
60%
70%
1998 2001 2004 2007
0%
10%
20%
30%
40%
50%
60%
70%
Sources Consulted for Information on Research Topics (Scale of 1 “Not at All” to 5 “Usually”)
Faculty
Grad
Undergrad
2.5 2.75 3 3.25 3.5 3.75 4 4.25 4.5
BibliographicDatabases
Library Catalog
Open Internet RefSource
Open Internet Search
Sources Consulted for Information on Research Topics (Scale of 1 “never” to 5
“usually”)
3
3.25
3.5
3.75
4
4.25
4.5
Health Sci BioSci Phy Sci-Engin Hum-Soc Sci
Bib Databases Open Internet Library Catalog
I wish the interface between scholar and UW libraries was better. I want to search with Scholar, but use UW's credentials to access full-length articles. Right now, there are a lot of intermediary pages I need to visit. Bioengineering grad student
Libraries Contribution to… (Scale of 1 “Minor” to 5 “Major”)
3 3.25 3.5 3.75 4 4.25 4.5 4.75
Academic success
Efficient use of time
Finding info in new orrelated areas
Keeping current inyour field
Being a moreproductive researcher
Grad Faculty
Takeaways after 16 Years
• Undergraduates– Library as Place– Work, meet, learn, live
• Graduate/professional students– Access to information and
services– How can the library save
me time?
• Faculty– Collections (physical and
virtual)
Innovative Responses
• Extend hours in Undergraduate Library (24/7)• Create more diversified student learning spaces• Privilege electronic journals • Enhance usability of discovery tools and
website• Provide standardized service training for all staff• Stop activities that do not add value • Consolidate and merge branch libraries• Change/reallocate collections budget and
staffing• Improve librarian liaison program to academic
areas
WorldCat Local
• Discovery to delivery challenge• One box, one stop shopping
– Formats integrated– UW holdings, regional consortium, WorldCat, 50 million articles– User doesn’t need to know how we do things internally
• Components– Content– Shared platform– Interoperability with local services
• Circulation, ILL, OpenURL
– Common marketing
Results to Date
• 20% increase in use of local collections
• 60% increase in borrowing from regional consortia
• 114% increase in world-wide interlibrary loan
• 200-275,000 searches per month
• WCL is our #2 ranked OpenURL origin and growing
UWired
• Established in 1994• Multiple partners• Teaching, learning,
and technology• Information smart
global citizens
Successes
• Collaboratories• Center for Teaching• Learning Commons• Technology-
enhanced spaces• Course tools• Workshops• MyUW• ONTECHNews
The Challenge: Maintain Relevancy and Centrality
Faculty 4.25
4.44
4.33 4.33
Faculty 4.56
Undergrad 3.97 3.99
4.22
4.32 UW Seattle UG 4.36
4.34
4.26
4.11Grad 4.18
Grad 4.36
3.8
3.9
4
4.1
4.2
4.3
4.4
4.5
4.6
1995 1998 2001 2004 2007
3.8
3.9
4
4.1
4.2
4.3
4.4
4.5
4.6
Overall Satisfaction by Group 1995-2007