Post on 11-Nov-2014
description
transcript
Strategic Collection Management through Statistical Analysis
Stephanie H. Wical
University of Wisconsin – Eau Claire
&
Hans Kishel
University of Wisconsin – Eau Claire
NASIG 2012
Methodology
• Identified academic libraries in Wisconsin using a directory maintained by Wisconsin Association of Academic Libraries.
• Surveyed academic librarians who were believed to have some role (if not sole responsibility) for electronic resource management.
• Interviewed academic librarians (who were a subset of the group who responded to the survey).
NASIG 2012
Demographics of the Group
The group included libraries in public and private colleges and universities, technical colleges, two-year colleges and for-profit colleges and universities in the state of Wisconsin or academic libraries that provide library service to residents of the state of Wisconsin.
NASIG 2012
Response Rate
• 139 surveys distributed via e-mail• 62 completed survey responses• 28 completed phone interviews
45% Response Rate for the Survey as of May 25, 2012
NASIG 2012
Survey Question: What types of library electronic resources statistics--if any--do you currently collect?
(Choose all that apply.)
Answer Responses PercentageSearches 54 87%Sessions 46 74%
Full Text Downloads 41 66%Cost-per-use 25 40%
IP Addresses that access resources 3 5%Simultaneous Users 9 15%
Turn-aways 27 44%
Times authors at your college or university are cited in resources 1 2%Times authors at your college or university cite resources 2 3%
Scholarly Impact 1 2%Not sure 3 5%
None 1 2%Other (Please specify.) 6 10%
N=62
NASIG 2012
Survey Question: What measure or measures do you consider when evaluating electronic resources in your
library? (Choose all that apply.)
Answer Response PercentageSearches 53 85%Sessions 39 63%
Full Text Downloads 38 61%Cost-per-use 37 60%IP Addresses 1 2%
Simultaneous Users 19 31%Turn-aways 21 34%
Times authors at your college or university are cited in resources 3 5%Times authors at your college or university cite resources 3 5%
Scholarly Impact 20 32%Not sure 2 3%
None 1 2%Other (Please specify.) 7 11%
N=62
NASIG 2012
A Comparison of Collected and Considered
Searc
hes
Sessi
ons
Full T
ext D
ownloads
Cost-per-
use
IP Addresses
that
access
reso
urces
Simulta
neous U
sers
Turn
-away
s
Times
authors
at yo
ur colle
ge or u
niversi
ty are
cited
in re
source
s
Times
authors
at yo
ur colle
ge or u
niversi
ty cit
e reso
urces
Scholar
ly Im
pact
Not sure
None
Other (Plea
se sp
ecify.
)0
10
20
30
40
50
60
ConsideredCollected
N=62
NASIG 2012
Searches, Sessions, Full Text Downloads and Turnaways
NASIG 2012
Survey Question: How frequently do you consider your measure or measures?
Never2%
Once a year39%
During the fall and spring semesters16%
Quarterly6%
Monthly16%
Not sure5%
Other (Please specify.)16%
N=62
NASIG 2012
Interview Question: Do you look at cost-per-use for electronic resources?
N=28
Yes68%
No32%
NASIG 2012
IP Addresses That Access Resources
This measure might be important down the road as departments outside of the library might have to share costs for resources.
NASIG 2012
Times Cited or Times Citing
Collecting this type of information was not a high priority for any of the librarians we talked to.
NASIG 2012
Scholarly Impact
• Scholarly impact is not something one routinely tracks, but it is something that is valuable when considering adding a new resource.
Image from Experimental Physiology at http://ep.physoc.org/site/misc/author.xhtml
NASIG 2012
Interview Question: What measure or measures do you consider when evaluating electronic resources in your library?
Sessi
ons
Searc
hesFT
-DL
Cost-per-
use
Turn
AwayPric
e
Duplication
Page V
iews
Click Th
rough
Releva
nce to pro
grams
Overla
p
Ease
of use
Curricu
lum
Coverag
e
Review
s
Authority
Platform
Quality
Usage S
tats
0%
10%
20%
30%
40%
50%
60%
70%
N=28
NASIG 2012
Interview Question: Why do you use the measure or measures you use to evaluate electronic resources?
Budget We have always done it that way / It is what
we get
To get idea of what students are using
Renewal To be able to compare
0%
5%
10%
15%
20%
25%
30%
35%
N=28
NASIG 2012
Interview Question: Do you feel that the measure or measures you use are adequate or do you feel that they do not provide relevant information?
Yes50%
So-So29%
No21%
N=28
NASIG 2012
People who were not satisfied with usage statistics indicated:
• Not all of them are counter compliant.• Could always use more data.• Sometimes usage statistics do not account for a lot
of variables.• It would be nice to find a measure that can be looked
at across vendors.• They are not relevant at all; statistics are a joke.• We are drowning in data.
NASIG 2012
Where one library may see value in a set of statistics, another library may need something more.
---Schufreider & Romaine (2008)
NASIG 2012
Survey Question: What tool or tools do you use to gather statistics for evaluating electronic journals, databases and/or
electronic books? (Choose all that apply.)
Answers Number Using Tool PercentagesSFX Ustat 5 8%
ERMes (developed at UW- La Crosse) 8 13%Scholarly Stats 1 2%
Serials Solution 360 Link 4 7%Serials Solutions 360 Resource Manager 5 8%
EBSCO ERM Essentials 4 7%EBSCO LinkSource 7 11%
Ex Libris’ Verde 1 2%Innovative ERM 1 2%
Open ERM 1 2%OCLC Worldcat Link Manager 2 3%
CORAL ERM 1 2%Excel spreadsheet or other spreadsheet software 40 66%
Access database or other database software 10 16%Not sure 10 16%
None 2 3%Other (Please specify.) 15 25%
N=61
NASIG 2012
Finding
66% are using Excel in addition to another tool
Q6 What tool or tools do you use to gather statistics for evaluating electronic journals, databases and/or
electronic books? Excel spreadsheet or other spreadsheet software
SFX Ustat
ERMes (develop
ed at UW- La Crosse)
Scholarly Stats
Serials Solution 360 Link
Serials Solutions
360 Resource Manager
EBSCO ERM
Essentials
EBSCO LinkSourc
eEx Libris’
VerdeInnovativ
e ERMOpen ERM
OCLC Worldcat
Link Manager
CORAL ERM Not sure
Other (Please specify.)
Excel spreadsheet or other spreadsheet software 4 7 1 3 2 4 6 1 1 1 1 1 1 7
N=61
NASIG 2012
Survey Question: Are you able to compare statistics across vendors and platforms?
Yes60%
No11%
Not sure16%
Other (Please spec-ify.)13%
N=62
NASIG 2012
Survey Question: Have you taken advantage of the Standardized Usage Statistics Harvesting Initiative (SUSHI)
protocol to obtain usage reports?
Yes7%
No57%
Plan to do so in the next two years
13%
Not sure16%
Other (Please specify.)7%
N=61
NASIG 2012
Survey Question: How important are usage statistics in the decision to renew or cancel electronic resources?
Very unimportant8% Unimportant
2%Neither unimportant nor important (neu-
tral)16%
Important39%
Very important35%
N=62
NASIG 2012
Survey Question: Have you ever canceled an electronic resource because it had low use?
Yes81%
No8%
Not sure11% N=62
NASIG 2012
“How important are usage statistics in the decision to renew or cancel electronic resources?”
&“Have you ever canceled an electronic resource
because it had low use?”
How important are usage statistics in the decision to renew or cancel electronic
resources?Have you ever canceled an electronic resource because it had low use? Grand Total
Yes NoNot sure
Very unimportant 3 2 5
Unimportant 1 1
Neither unimportant nor important (neutral) 6 3 1 10
Important 20 1 3 24
Very important 21 1 22
Grand Total 50 5 7 N=62
NASIG 2012
What do you collect and how important is it?
Q3 What types of library electronic resources statistics--if any--do you currently collect? (Choose all that apply.)
Q14 How important are usage statistics in the decision to renew or cancel electronic resources? Searches Sessions
Full Text Download
sCost-per-
use
IP Addresses
that access
resources
Simultaneous Users
Turn-aways
Times authors at
your college or university are cited
in resources
Times authors at
your college or university
cite resources
Scholarly Impact Not sure None
Other (Please specify.)
Very unimportant 4 4 4 4 4 1 1 1Unimportant 1 Neither unimportant nor important (neutral) 8 7 6 2 1 3 1 Important 21 19 16 8 2 7 11 1 1 1 3Very important 21 16 15 11 1 1 9 1 2
Grand Total 54 46 41 25 3 9 27 1 2 1 3 1 6
NASIG 2012
Using Statistics
What will you do with usage statistics?– Make changes and improvements– Request additional resources based
upon data
NASIG 2012
Interview Question: What do you think that you should be doing with electronic resource usage statistics?
Marketing12% What is getting used
12%
Bang for the Buck15%
More indepth12%
Communication19%
Renewal/informed decisions
19%
Better organized / SUSHI12%
N=28
NASIG 2012
Survey Question: Do you plan to collect additional statistics in the next two years?
Yes37%
No10%
Maybe; it will depend on cir-
cumstances (e.g. budget or
staffing)42%
Not sure11%
N=62
NASIG 2012
Sharing Usage Statistics
• Who needs to know about usage statistics?– Identify stakeholders– What will the reports look like?– What is the right amount of information?– Should the reports look different for
different people?– Should an executive summary be
written for an administrator?
NASIG 2012
Survey Question: Are usage statistics for library electronic resources reported outside the library?
N=62
Yes50%
No24%
Not sure26%
NASIG 2012
Survey Question: Who receives regular electronic resources statistical reports from the person or people who collect them?
(Choose all that apply.)
Answers Responses PercentagesElectronic Resources Librarian 10 16%Periodicals Librarian 7 11%Technical Services Librarian 5 8%Director / Dean 30 48%Administrator who supervises the director or dean of the library 7 11%Reference Librarian 10 16%Systems Librarian 5 8%Everyone in the library 13 21%Everyone on the e-resources team, work group or task force 4 6%Anyone who reads the annual report 5 8%Anyone who reads the library newsletter 0 0%Anyone who reads the library blog, Facebook page, or twitter feed 0 0%No one 7 11%Not sure 4 6%Other (Please specify.) 16 26%
N=62
NASIG 2012
Survey Question: Who collects electronic resources statistical reports at your library? (Choose all that apply.)
Answers Responses PercentagesElectronic Resources Librarian 17 27%Periodicals Librarian 7 11%Technical Services Librarian 6 10%Director / Dean 13 21%Reference Librarian 6 10%Systems Librarian 8 13%Paraprofessional 13 21%Student Workers 2 3%We don’t collect them at our library---they are collected on a system or consortial level 10 16%Our vendor collects them for us 11 18%We do not collect them at all 1 2%Not sure 0 0%Other (Please specify.) 10 16%
N=62
NASIG 2012
Survey Question: Is collecting electronic resources statistical data something that is written into one or
more job descriptions?
N=62
Yes37%
No44%
Not sure18%
Other (Please spec-ify.)2%
NASIG 2012
Interview Question: What would you consider a best practice for collection management statistics for electronic resources?
Answer # of responses %No Idea 7 25%
ERM 2 7%SUSHI 5 18%
Collect Regularly 1 4%Check them to make sure they are right 1 4%
Big Picture thinking 2 7%Compare over time 3 11%
Feedback 2 7%Review Regularly 1 4%How students use 1 4%
Cost per use 1 4%COUNTER 2 7%Use them 2 7%
N=28
NASIG 2012
Interview Question: How do you prioritize uses of the electronic resources usage statistics you collect? Or what is most
important?
Answers Number of Responses PercentagesWhat getting used 9 32%
Cost per use 1 4%Renewal 6 21%
Annual Report 5 18%Turnaways 1 4%
FT-DL 4 14%Search 5 18%
Course integration 2 7%Sessions 1 4%Budget 4 14%
Faculty Requests 1 4%Vendor Data 2 7%ACRL Reports 1 4%
Don't 1 4%Advertising 1 4%
Bang for Buck 2 7%Look for Changes/Problems 2 7%
N=28
NASIG 2012
Assessment Process
NASIG 2012
Survey Question: Has there been any attempt at your library to incorporate usage data into assessment of library services?
Yes48%
No35%
Not sure16%
N=62
NASIG 2012
Interview Question: How do you determine if usage statistics are evaluated and consulted successfully?
Don't59%
If there is a problem then something is
wrong15%
Feedback26%
N=28
NASIG 2012
Statistics can play a more progressively important role in determining the return on investment with library dollars.
---Beals & Lesher (2010)
NASIG 2012
80/20 Distribution
Thomas Nisonger (2008) states, “The basic 80/20 pattern provides a valid approach to operationalizing the core journal concept and is applicable to collection management decision making” (p. 78).
Although Nisonger (2008) admits that the percentages do not match the 80/20 rule exactly, should we make sure that the majority of our budget is going into resources that get the majority of use?
NASIG 2012
Quantitative Data
– Cost-per-use– Number of disciplines– Historical usage data (including five year trends)– Interlibrary loan data– Impact factors
Source: Hulbert, Roach & Julian (2011)
NASIG 2012
Qualitative Data
– Experience or the gut feeling– User feedback– Reviews– The dust factor for print resources
Source: Hulbert, Roach & Julian (2011)
NASIG 2012
Decisions and planning up front are often the most time-consuming part of the process.
--Hulbert, Roach & Julian, 2011
NASIG 2012
Case Study INancy Beals of Wayne State University created a “Stats Master” spreadsheet and training program for selectors (Beals & Romaine, 2010).
NASIG 2012
Case Study IILinda Hulbert & Dani Roach of University of St. Thomas Libraries developed an “Electronic Resources Review Checklist” and a way to “connect the silos” to contribute to the university’s overall culture of assessment (Hulbert, Roach & Julian, 2011)
NASIG 2012
Case Study IIIMary Ann Trail and Kerry Chang-FitzGibbon describe how the Library of the Richard Stockton College of New Jersey addressed a $29,000 shortfall using an overlap analysis report (Trail, Chang-FitzGibbon & Wishnetsky (2012).
NASIG 2012
Case Study IVBryan Vogh and Hans Kishel (2011)at the University of Wisconsin – Eau Claire charted the workflow necessary for adding a new database for the McIntyre Library to best utilize BP Logix.
NASIG 2012
Case Study V
Carol Tenopir (2010) looks at ROI of the value of e-journals to grants income.
• “I could not submit as many grants. With grant funding at 4-6% of submitted proposals I would not have achieved my current funding level.” (U.S. University)
• “I guess on average the online access saves me more than 10 hours per week.” (Western European Research Institute)
(p. 44).
NASIG 2012
Case Study VI-?
What You or Your Colleagues are Doing
(Audience Provides At Least One Example.)
NASIG 2012
References
Bob Schufreider & Sion Romaine (2008): Making Sense of your Usage Statistics, The Serials Librarian, 54:3-4, 223-227, from http://dx.doi.org/10.1080/03615260801974164
Bryan Vogh & Hans Kishel (2011): E-forms: Making Workflows Work, UMWUG Conference in Fargo, ND, October 17, 2011, from http://people.uwec.edu/voghbs/UMWUG_E-forms_Handout.pdf
Carol Tenopir (2010): Measuring the Value of the Academic Library: Return on Investment and Other Value Measures, Serials Librarian, 58:1-4, 39-48, from http://dx.doi.org/10.1080/03615261003623005
Linda Hulbert, Dani Roach & Gail Julian (recorder)(2011): Integrating Usage Statistics into Collection Development Decisions, The Serials Librarian, 60:1-4, 158-163, from http://dx.doi.org/10.1080/0361526X.2011.556027
Mary Ann Trail, Kerry Chang-FitzGibbon & Susan Wishnetsky (recorder)(2012): Using Assessment to Make Difficult Choices in Cutting Periodicals, The Serials Librarian, 62:1-4, 159-163, from http://dx.doi.org/10.1080/0361526X.2012.652931
Nancy Beals & Marcella Lesher (2010): Managing Electronic Resource Statistics, The Serials Librarian, 58:1-4, 219-223, from http://dx.doi.org/10.1080/03615261003625844
Sarah Glasser & Michael A. Arthur (recorder) (2011): When Jobs Disappear: The Staffing Implications of the Elimination of Print Serials Management Tasks, The Serials Librarian, 60:1-4, 109-113, fromhttp://dx.doi.org/10.1080/0361526X.2011.556447
Thomas E. Nisonger (2008): The “80/20 Rule” and Core Journals, The Serials Librarian, 55:1-2,62-84, from http://dx.doi.org/10.1080/03615260801970774
NASIG 2012
Thank You!
Stephanie H. WicalAssistant ProfessorPeriodicals and Electronic Resources LibrarianMcIntyre LibraryUniversity of Wisconsin – Eau Claire105 Garfield AvenueEau Claire, WI 54702(715) 836-3508wicalsh@uwec.edu
Hans F. KishelAssistant ProfessorResearch and Instruction Librarian for ScienceMcIntyre LibraryUniversity of Wisconsin – Eau Claire105 Garfield AvenueEau Claire, WI 54702(715) 836-2959kishelhf@uwec.edu