Post on 26-Aug-2020
transcript
PDA MetricsHow I Learned to Stop Worrying and Trust My Users
Andrew See Teresa Hazen Jason Dewland
Who is this guy?
Andrew SeeHead of User Services
Northern Arizona University
Cline Library
• Formerly at the University of Arizona Libraries. • Member of UAL’s PDA Metrics group• Former UAL Acquisitions Manager• Co-author: Patron driven acquisitions -
Determining the metrics for success. Library Resources and Technical Services
Forces for Change
ODID (on demand
information delivery
Declining Budget
Return on Investment
User Focused
Collection
Changing Role of
Librarians
PDA at the UA
• ODID Project began in the Summer of 2011
• ODID acts as the main driver of our collection.
• More than 142,000 discoverable titles have been added to our collection.
• Print: direct order links to our vendor are embedded in our Web OPAC and discovery layer. Purchases are delivered to our library shelf-ready and on-hold for the customer.
• Electronic: Seamless to the user. Purchases are triggered after a set number of uses
Attack of the what ifs
Losing control of your
collection?
Sky rocketing budget?
Unbalanced purchases?
Set yourself up for success
Pre-PDA
2010
(1 year)
Post-PDA
2011-present
(4 years)
Books purchased 13,723 18,121
Cost of Purchase $1,116,803 $1,743,146
Cost per Year $1,116,803 $435,787
Unpurchased
Records exposed 0 124,000
% purchased from
exposed titles 0% 12.75%
Create your profile
• All PDA titles are current
scholarly materials
• No textbooks
• No popular fiction
• No manuals
Data supplied by Stephen Bosch
Materials Budget, Procurement & Licensing Librarian
University of Arizona Libraries
Research Services Librarian
Library Analyst
Library Information Associate:
Acquisitions
Library Information
Associate: Library Infrastructure
Materials Budget, Procurement, &
Licensing Librarian
ODID Metrics Project Team
Charge:To coordinate the design and implementation of the data gathering processes to evaluate the ODID programs' effectiveness
Deliverables:An efficient set of metrics to help determine:• The quality of resources exposed to
library customers• Amount of use of those resources• Cost Effectiveness of the program• Customer satisfaction of the program
MetricsFinancial
• Cost Per Use (CPU)
• CPU by LC Class
• Total expenditures of program
Patron
• CustomerSatisfaction
• Customer Behavior
• User Experience
Performance
• Average Delivery Time of print resources
• eBooks allowing adequate download options
Usage
• Circulation of Print books
• Use of eBooks
• How delivery time of print resources affects usage
Resource
• Understanding of resource pool and how that drives user behavior
• Percent of resources purchased from available options
• How currency of resources affects usage
For a complete list and definition of metrics:
Dewland, J. C., & See, A. (2014). Notes on operations: Patron driven
acquisitions - Determining the metrics for success. Library Resources and
Technical Services, 59, 1, 13-23.
UA’s Print PDA ProgramUser and staff perspectives
Who is this gal?
Teresa HazenInterim Head of Delivery, Description, &
Acquisitions
University of Arizona Libraries
• Technical services librarian• Currently manages UA Libraries PDA programs
Print PDA Record in ILS Public Catalog
Authenticating
Coutts API captures information about the user based on the NetID so that a hold can be
eventually placed on the item for the customer. Information gathered via the API is imbedded in a
961 of the MARC record-
Order Placed by User
Status Changes in ILS
Success!
• User gets an email when book has
been processed and is on the hold
shelf ready to be picked up.
• The average is 10 business days
Some Steps in the Process for Library Staff
• Link in catalog is changed (daily report)
• MARC record file arrives with patron information in local 9xx MARC fields
• Staff overlays a bib. record, creates item, and edits order record
• Staff manually places a hold in ILS for user
Actual workflow somewhat more complicated…
Print PDA: Mapping the Workflow
Gathering the DataPulling data from your ILS for analysis and assessment
Used our ILS to Gather and Export the Data
• We are on Millennium 2014 2.0.1_2
• Used Create Lists functionality to gather records for exporting. Created
multiple lists and then exported certain fields in tab delimited format for
data normalization and analysis.
Limiters
�We only wanted to look at monographs regardless of format since we have a legacy of print but a recent history of almost entirely electronic.
�Made the decision to include only three library branches- our Main Library, Science and Engineering Library, and Fine Arts Library.
�Attempted to the best of our abilities to exclude government document loads (ex. MARCIVE).
Limiters, cont.
• We limited to monographs or eBooks—
• Material type= a (book) AND w (local code for ebook)
• Though we included only three libraries, this meant wading through dozens of location codes to include/exclude
• Ex.: Include Oversized
• Ex.: Exclude Information Commons Reference Desk
Yet More Limiters• Created multiple files each based on limiting to specific date ranges based on when
we started our patron driven acquisitions print and electronic programs:
• Pre-PDA: 01-01-2006 – 12-31-2009
• Post-PDA 01-01-2010 – 12-31-2014
• Post-PDA, no print PDA 01-01-2010 – 12-31-2014
• Weeded out as many government documents as possible by identifying a code in the ILS for all MARCIVE loaded documents (a bcode3, which is a locally defined code)
Creating the Print PDA File
• After creating the bigger monograph file from 2010-2014, we pulled out the
purchased print PDA:
• Ran a search on the 2010-2014 monograph list and included—
• The presence of an order record linked to the bibliographic record
• A 500 MARC note field indicating the record used to be a selection PDA
Creating the Export Files
We wanted to export out call number for analysis, knowing that some of our ebook collections don’t have a call number. We wanted location and material type as well as total number of checkouts as possible data points.
Fields to export included:
• Call number
• Location
• Material type
• For print PDA titles:
• Total checkouts (from item record)
• Estimated price (from order record)
Output the Export File & Future Issues
• Using Data Exchange functionality in Millennium, we output the file as a tab delimited for normalization and
analysis.
___________________________________
• With the introduction of discovery tools and next generation library management systems, there will be big
changes in the way we mine data from our curated content.
• As we move away from being in control of the creation of metadata to describe our content, we must push for
better metadata, such as LC call numbers, better identifiers of packages and collections.
• It’s really important to think in advance of actions you can do that help you work better with the data for future
analysis.
• Don’t ever get rid of call numbers
• Make use of local fields (9XX) to identify collections
• When using overlay functionality in your ILS, be sure to preserve metadata such as vendor control numbers
What the Data Tells UsPDA program’s real influence on the collection
Who is this guy?
Jason DewlandAssistant Librarian, Research & Learning
Department
University of Arizona Libraries
• Business & Tech Transfer Librarian• Member of UAL’s PDA Metrics group• Former UAL Acquisitions Manager• Co-author: Patron driven acquisitions -
Determining the metrics for success. Library Resources and Technical Services
Summary of Purchases 2006 -2009
• Added over 440,000 monographs during the time period.
• Approximately 110,000 were hard copy books.
• The rest were eBooks, government documents, pamphlets, and other
materials.
• We only have LC classification data for hard copy books, eBooks were
stripped of this data.
Summary of Purchases 2010 to 2014
• PDA made up only a small percentage (16.7%) of our purchases over this time period.
• PDA resulted in 7,128 monographs and 64,335 eBooks purchased.
• As a percent it was a similar pattern at the Sub - LC Classification pre & post PDA program.
• One hole is that eBooks were stripped of their LC number until 2011. Since the beginning of PDA approximately 16,965 (26.0%) eBooks purchased with no LC number.
• One difference in the shape of the collection – P – Language and Literature – 10% drop in percentage of purchases after implementation of PDA
Dewland, J., Elliott, C., Hazent, T., & See, A. “Patron Driven Acquisition:
Has it Changed Our Collections?” [Poster]. American Library Association Annual Conference,
San Francisco, CA., June 30th, 2015.
P – Language & Literature2010-2014 Comparison by Format
Hard Copy eBook
2010-2014 Count of P
Books Purchased
Percent of P Books
Purchased
2010-2014 Count of P
Books Purchased
Percent of P Books
Purchased
PE 107 8.4% PE 203 2.5%
PN 337 26.5% PN 2290 27.8%
PQ 54 4.2% PQ 394 4.8%
PR 190 14.9% PR 2007 24.4%
PS 265 20.8% PS 1388 16.9%
Total 1271 Total 8236
Red = Larger percentage of eBooks purchased than hardcopies.
Black = Larger percentage of hardcopies purchased than eBooks.
P – Language & Literature2006-2009 v. 2010-2014
2006-2009PDA
2010-2014
P 1167 3.6% P 1094 11.5%
PA 688 2.1% PA 287 3.0%
PB 52 0.2% PB 23 0.2%
PC 327 1.0% PC 58 0.6%
PD 16 0.0% PD 14 0.1%
PE 383 1.2% PE 310 3.3%
PF 125 0.4% PF 17 0.2%
PG 750 2.3% PG 115 1.2%
PH 25 0.1% PH 9 0.1%
PJ 2243 6.9% PJ 146 1.5%
PK 336 1.0% PK 54 0.6%
PL 2247 6.9% PL 247 2.6%
PM 148 0.5% PM 48 0.5%
PN 4076 12.5% PN 2627 27.6%
PQ 7614 23.3% PQ 448 4.7%
PR 4924 15.1% PR 2197 23.1%
PS 5620 17.2% PS 1653 17.4%
PT 1907 5.8% PT 156 1.6%
PZ 22 0.1% PZ 4 0.0%
Total 32670 Total 9507
P – Language & Literature2006-2009 v. 2010-2014
2006-2009 2010-2014 Change
PG 750 2.3% PG 115 1.2% 635 -1.1%
PJ 2243 6.9% PJ 146 1.5% 2097 -5.3%
PL 2247 6.9% PL 247 2.6% 2000 -4.3%
PQ 7614 23.3% PQ 448 4.7% 7166 -18.6%
PT 1907 5.8% PT 156 1.6% 1751 -4.2%
P 1167 3.6% P 1094 11.5% 73 7.9%
PN 4076 12.5% PN 2627 27.6% 1449 15.2%
PR 4924 15.1% PR 2197 23.1% 2727 8.0%
P – Language & Literature
Non-PDA and PDA2010-2014 Non-PDA 2010-2014 Combined 2010-2014
P 1094 11.5% P 4817 9.2% P 5911 9.6%
PA 287 3.0% PA 1171 2.2% PA 1458 2.4%
PB 23 0.2% PB 80 0.2% PB 103 0.2%
PC 58 0.6% PC 385 0.7% PC 443 0.7%
PD 14 0.1% PD 73 0.1% PD 87 0.1%
PE 310 3.3% PE 1065 2.0% PE 1375 2.2%
PF 17 0.2% PF 113 0.2% PF 130 0.2%
PG 115 1.2% PG 824 1.6% PG 939 1.5%
PH 9 0.1% PH 41 0.1% PH 50 0.1%
PI 0 0.0% PI 1570 3.0% PI 1570 2.5%
PJ 146 1.5% PJ 596 1.1% PJ 742 1.2%
PK 54 0.6% PK 1190 2.3% PK 1244 2.0%
PL 247 2.6% PL 12243 23.5% PL 12490 20.2%
PM 48 0.5% PM 2268 4.3% PM 2316 3.8%
PN 2627 27.6% PN 5405 10.4% PN 8032 13.0%
PQ 448 4.7% PQ 7263 13.9% PQ 7711 12.5%
PR 2197 23.1% PR 5624 10.8% PR 7821 12.7%
PS 1653 17.4% PS 6065 11.6% PS 7718 12.5%
PT 156 1.6% PT 1327 2.5% PT 1483 2.4%
PZ 4 0.0% PZ 59 0.1% PZ 63 0.1%
Total 9507 Total 52179 Total 61686
P – Language & Literature
Non-PDA and PDA2010-2014 Non-PDA 2010-2014 Combined 2010-2014
P 1094 11.5% P 4817 9.2% P 5911 9.6%
PA 287 3.0% PA 1171 2.2% PA 1458 2.4%
PB 23 0.2% PB 80 0.2% PB 103 0.2%
PC 58 0.6% PC 385 0.7% PC 443 0.7%
PD 14 0.1% PD 73 0.1% PD 87 0.1%
PE 310 3.3% PE 1065 2.0% PE 1375 2.2%
PF 17 0.2% PF 113 0.2% PF 130 0.2%
PG 115 1.2% PG 824 1.6% PG 939 1.5%
PH 9 0.1% PH 41 0.1% PH 50 0.1%
PI 0 0.0% PI 1570 3.0% PI 1570 2.5%
PJ 146 1.5% PJ 596 1.1% PJ 742 1.2%
PK 54 0.6% PK 1190 2.3% PK 1244 2.0%
PL 247 2.6% PL 12243 23.5% PL 12490 20.2%
PM 48 0.5% PM 2268 4.3% PM 2316 3.8%
PN 2627 27.6% PN 5405 10.4% PN 8032 13.0%
PQ 448 4.7% PQ 7263 13.9% PQ 7711 12.5%
PR 2197 23.1% PR 5624 10.8% PR 7821 12.7%
PS 1653 17.4% PS 6065 11.6% PS 7718 12.5%
PT 156 1.6% PT 1327 2.5% PT 1483 2.4%
PZ 4 0.0% PZ 59 0.1% PZ 63 0.1%
Total 9507 Total 52179 Total 61686
P – Language & LiteraturePackage Purchases
2006-2009Combined
2010-2014
Difference in Titles
Purchased
P 1167 3.6% P 5911 9.6% 4744
PA 688 2.1% PA 1458 2.4% 770
PB 52 0.2% PB 103 0.2% 51
PC 327 1.0% PC 443 0.7% 116
PD 16 0.0% PD 87 0.1% 71
PE 383 1.2% PE 1375 2.2% 992
PF 125 0.4% PF 130 0.2% 5
PG 750 2.3% PG 939 1.5% 189
PH 25 0.1% PH 50 0.1% 25
PI 0 0.0% PI 1570 2.5% 1570
PJ 2243 6.9% PJ 742 1.2% (1501)
PK 336 1.0% PK 1244 2.0% 908
PL 2247 6.9% PL 12490 20.2% 10243
PM 148 0.5% PM 2316 3.8% 2168
PN 4076 12.5% PN 8032 13.0% 3956
PQ 7614 23.3% PQ 7711 12.5% 97
PR 4924 15.1% PR 7821 12.7% 2897
PS 5620 17.2% PS 7718 12.5% 2098
PT 1907 5.8% PT 1483 2.4% (424)
PZ 22 0.1% PZ 63 0.1% 41
Total 32670 Total 61686 29016
P – Language & Literature
Winners/Losers
2006-2009Combined
2010-2014
Difference in
Titles
Purchased
Difference in %
of Collection in
Class P
P 1167 3.6% P 5911 9.6% 4744 6.0%
PE 383 1.2% PE 1375 2.2% 992 1.1%
PL 2247 6.9% PL 12490 20.2% 10243 13.4%
PJ 2243 6.9% PJ 742 1.2% (1501) -5.7%
PQ 7614 23.3% PQ 7711 12.5% 97 -10.8%
PR 4924 15.1% PR 7821 12.7% 2897 -2.4%
PS 5620 17.2% PS 7718 12.5% 2098 -4.7%
PT 1907 5.8% PT 1483 2.4% (424) -3.4%
The PDA Haboob?
• Since 2010 the selection pool has grown by 92% for PDA
• The expenditures have grown 81%
• Continue to expand formats available for PDA
Gust-front dust cloud, called a Haboob, moving across the Llano Estacado toward Yellow House Canyon near the residential community of Ransom Canyon, Texas
Questions?