Assessing for Digital Library
ReuseInitial Findings from The Measuring Reuse
ProjectGenya O’Gara
Caroline MugliaSanti Thompson
Elizabeth Joan KellyAyla Stein KenfieldLiz Woolcott
Presentation Outline
2
I. Project Background II. Initial Survey
III. Focus Group Sessions IV. Use CasesV. Follow Up Survey
VI. Next Steps
Project Background
Santi Thompson
4
Santi ThompsonUniversity of [email protected] | @SantiThompson
Elizabeth Joan KellyLoyola University, New [email protected] | @ElizabethJelly
Caroline MugliaUniversity of Southern [email protected]
Genya O’GaraVirtual Library of Virginia, George Mason [email protected] | @genyao
Ayla Stein KenfieldUniversity of Illinois [email protected] | @TheStacksCat
Liz WoolcottUtah State [email protected] | @lizwoolcott
#digreuseProject Team
reuse.diglib.org
5
Previous Work and Current Support
Assessment Interest Group (AIG) Surveying the Landscape: Use and Usability Assessment of Digital Libraries (2015) white paper
2017 National Leadership Grant (LG-73-17-0002-17)July 01, 2017 – June 30, 2018
Functional requirements and use cases for a reuse assessment toolkit▸ Sustainable and vetted assessment
techniques▸ Methods to deepen understanding of users▸ Developing approaches and best practices to
communicate impactFinal Products▸ White paper▸ Grant Report▸ Project Data
Where to find deliverables▸ Website: https://reuse.diglib.org ▸ OSF Repository: https://osf.io/y5tjh
6
Project Deliverables
7Advisory
Board
Hannah FrostEmily GoreKrystyna K. MatusiakSandra PhoenixDorothea SaloAli Shiri
8
Assessing Use vs. Reuse
UseDiscovering and browsing objects in a digital repository, often described as “downloads” or “clicks,” without knowing the specific context for the use.Ex. Web analytics; CMS supplied analytics
ReuseHow often and in what ways digital repository materials are utilized and/or repurposed, knowing the context of the use.
Ex. Social media use; reverse image lookup
Our Process
9
Data Collection
And AnalysisFG
Initial S
urvey
Results
S
UC
Focus Group
Coded Data
Prioritized Reuse Assessment
Use Cases
Initial Survey
Initial Survey DetailsOpen:September-October
Software:Qualtrics
Questions:19 total10 “core”
Responses:409 total302 usable130 answered all core questions
Respondents:56% academic libraries10% public libraries8% museums(Remaining: archives; government libraries; historical societies; consortium; other)30% primarily serve under-represented groups
11
Use statistics12
Initial findings
Reuse statistics13
Initial findings
14
Because...
▹Improve digital collections services
▹Demonstrate the impact of collections on scholarship and research
▹Prove the return on investment
▹Improve collection selection decisions
Assessment Motivation
Use:Most common platforms:▹ Google Analytics▹ CMS Platform Stats
Most common data collected:▹ Number of visitors▹ Number of downloads▹ Number of clicks
Platforms and Methods
Reuse:Most common methods:▹ Social media metrics▹ Alert services▹ Reverse Image lookup
Most common data collected: ▹ Citations in scholarship▹ Published or replicated
in physical media
15
Use:
▹ No or new digital collections
▹ Lack of staff▹ Different
department’s purview
▹ Not a priority
Barriers
16
Reuse:
▹ Lack of methodology/system
▹ Lack of staff/time▹ Not an institutional
focus▹ Lack of
training/resources
Obstacles to Assessment
17
“Most of our understanding of reuse comes anecdotally. We
haven't figured out a good quantitative way to measure
reuse that isn't too labor intensive for us. This is a pain point, since
reuse is highly valued by stakeholders!
18
Focus Group Sessions
Genya O’Gara
20
Focus Group Type Sessions When Participants Topic
Group 1 In person 2 October 2017 9 Information Gathering
Group 1 Virtual 2 December 2017 7 Information
Gathering
Group 2 In person 2 February 2018 8 Technology & Standards
Group 2 Virtual 2 March 2018 7 Technology & Standards
Group 3 In person 1 April 2018 5 Cultural & Privacy
Group 3 Virtual 1 June 2018 4 Cultural & Privacy
Focus Group Participants
21
Focus Group Analysis
22
Document CodeGenerate
Use Cases
Examples of Reuse
23
▹ metadata aggregators
▹ digital exhibits▹ archiving websites▹ social media▹ commercializing
content▹ blogs/other online
publishing ▹ scholarly citations ▹ support cultural or
political statements▹ games▹ commentary/
annotation▹ remixes▹ presentations
▹ digital humanities project/research
▹ virtual reality environment
▹ genealogical research
▹ institutional history ▹ educational lesson
plans/instruction▹ research for historic
preservation ▹ Wikipedia citations▹ documentary footage
and/or research▹ workshop and
training sessions▹ marketing materials
Technology“Sticky” metadata, Data dashboards, Unique identifiers for all digital objects
SharingCollections data, success and failures, usage/reusage data repositories
Standards & Training...for gathering data, turning data into impact statements and stories, in using data to make decisions, in weighing the importance of data, in respecting user privacy
24 Needs
Reuse Assessment Use Cases
Use Cases
Data Collection, Analysis, and ReportingUse cases that address how reuse data is being assembled, interpreted, and communicated to various audiences and stakeholders.
Privacy, Rights Management, and EthicsUse cases that address the assessment of reuse through legal, ethical, and culturally-aware perspectives
Collection Development
Use cases that address how to assess aspects of a digital collection based on reuse data.
26
Data Collection, Analysis, and Reporting (1 of 2)
The practitioner:
▹ assesses quality and quantity of items reused from peer institutions for benchmarking
▹ wants to tell stories of impact with the reuse data they’ve captured and tailor them to various audiences/stakeholders
▹ generates reports of collection item reuse for analysis
27
28
Data Collection, Analysis, and Reporting (2 of 2)
The practitioner:▹ employs qualitative and quantitative methods
to collect reuse data▹ understands how content is being reused in a
variety of contexts by various audiences.▹ can find best practice documentation for
collecting, analyzing, and reporting re-use data
▹ is able to articulate the similarities and differences between use and reuse
29
Privacy, Rights Management, and Ethics (1 of 2)
The practitioner:▹ wants to know who their users are without
violating user privacy▹ can identify appropriative reuses of materials
from marginalized communities▹ can identify platforms/venues where reuse
content is occurring for commercial or marketing gain
30
Privacy, Rights Management, and Ethics (2 of 2)
The practitioner:▸ wants to enable/encourage attribution of
materials from their collection in various reuse contextsworks
▸ provides ability for marginalized communities to report incidents of reuse that the community perceives as culturally exploitive
▸ uses a code of ethics/conduct for assessing the reuse of content from diverse communities
31
Collection Development
The practitioner:▸ assesses quality and quantity of items reused to
identify strengths and gaps in digital collections▸ assesses quality and quantity of items reused to
inform digitization projects and priorities▸ update existing access records with value-added
content made by re-users▸ develop outreach services based on reuse data▸ implement DAMS/DLS with specific functionality
for enabling reuse
Follow Up Survey
Caroline Muglia
14 questions7 demographic, 7 topical questions
54% Academic Largest response groupNext largest: 18% public libraries
80% completion232 completed responses, 289 total
33
Comparison to Initial Survey
Total Questions Usable Responses
Respondents Under- served populations
Academic Public Museums All Others
Initial 19 302 56.3% 10.3% 8% 25.3% 30%
Follow-Up 14 232 54% 18% 6% 22% 51%
34
35
Barriers to Digital Collection Assessment
weighted and averaged(n=226, range 5.01-7.03)
Use Cases
▹ Data Collection, Analysis, and Reporting
▹ Collection Development
▹ Privacy, Rights Management, & Ethics
36
37
Use Cases:Data Collection, Analysis, Responsibility
weighted and averaged(n=218, range 3.01-4.37)
38
Use Cases:Collection Development
weighted and averaged(n=184, range 1.45-1.55)
39
Use Cases:Privacy,Rights management, Ethics
weighted and averaged(n=220, range 2.65-3.66)
40
Functionalities for a Reuse Assessment Toolkit
weighted and averaged(n=223, range 7.45-13.09)
Next Steps
Caroline Muglia
42
Spread the word▹ Setting a Foundation for Assessing Content
Reuse: A White Paper From the Developing a Framework for Measuring Reuse of Digital Objects project.
▹ Performance Measurements & Metrics, “Barriers and Solutions to Assessing Digital Library Reuse: Preliminary Findings”
43
Continuing to explore
▹ Recontextualization & controversial contexts of reused digital assets
▹ Platforms must keep up with practitioners
▹ Demonstrate value and impact▹ Privacy and reuse
44
Applied for another grant
▹ Focus on building toolkit based on use cases and functional requirements
THANKS!What are your questions?
reuse.diglib.org
45
CREDITS
Special thanks to all the people who made and released these awesome resources for free:▹ Presentation template by
SlidesCarnival▹ Photographs by Unsplash
46