PublicServiceforLarge‐ScaleDigitalCollec5onsJohnMarkOckerbloom(Penn)
LeslieJohnston(LibraryofCongress)ChrisPowellandJeremyYork(Michigan)
DLFforumworkingsessionOctober31,2011
It’sallaboutcollabora5on• Wecollaboratewithusers
• Wecollaboratewitheachother
• Adis5nc5vequalityoflibraries
FromaphotobyColleenMcMahon,CC‐BYhQp://www.flickr.com/photos/gatz125/5134393346/
Issueswe’lldiscuss
• Whatdigitalpublicserviceaccomplishes• Implemen5ngdigitalpublicservices• Evalua5ngandscalingupdigitalpublicservices
• We’llpresentquicktakesontheseissues,aswe’vedealtwiththeminourcollec5ons,inthefirsthalfofthissession
• Thesecondhalfisyourturntodiscuss,andaskandanswerques5ons
Differentcollec5ons,differentscales
• MLibraryDigitalCollec5ons– PresentedbyChrisPowell
• HathiTrustDigitalLibrary– PresentedbyJeremyYork
• Lib.ofCongressDigitalCollec5ons&Services– PresentedbyLeslieJohnston
• TheOnlineBooksPage– PresentedbyJohnMarkOckerbloom
Qs:Goalsofdigitalpublicservice
• Howdoyoufindoutwhoyourusersare,andwhattheymostneed?
• What’sthemostusefulthinguserscandoforyou,andhowcanyoubestgetthemtodoit?
• Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?
• Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?
Qs:Implemen5ngdigitalpublicservice
• Doyouhaveservicequalitygoalsandbenchmarks?Howdoyouevaluatethem?
• Whodoesthework?Howdoyouallocatelabor,anddotriage?
• Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?
• Hasuserfeedbackpromptedyoutochangeyourdesignorwhatyoudo?How?
Qs:Evalua5ngandscalingupdigitalpublicservice
• Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdatadoyoufindmosteffec5ve?
• Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?
• Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?
UniversityofMichiganDigitalLibraryProduc5onService
ChrisPowell
Lotsofstuff
• Variousformats:– Con5nuoustoneimagecollec5ons
– Full‐textcollec5ons:encodedXMLandpageimages
– Non‐MARCmetadatacollec5ons– EAD‐encodedfindingaidcollec5ons
• Varioussources• Variousaccessmodels
Lotsofques5ons
• Viaemailfrom1996‐2004• MovedtoFootprintsinJune2004
• Hardtoquan5fytheearlydays– Noformalprotocol– NoaQempttopreservetheques5onsoranswers– IntermiQentlyincludedinoveralllibraryreferencesta5s5cs
Footprints
• Centralizeemailfromusersofouronlineresources
• Replytopeoplewhoprovidecontactinforma5on
• Actonthosethatwecan,evenwithoutcontactinforma5on
• Deletethosewecannotfollowupon• Assignedtothoseresponsibleforthecontent
Afewnumbers
• Justover5000“5ckets”inthesystem,openandclosed
• 25,0005cketshavebeendeleted– Spam– Anonymousandnotac5onable
• Roughly650ofthe50005cketsarecategorizedasMBooksorHathiTrust
• MovedHathiTrusttoitsownsysteminMarch
Whoareweserving?
• UniversityofMichiganaffiliates• Affiliatesofuniversi5eswhosubscribetomaterialswehost
• Otherlibrariansandacademics
• Otherstudents• Thegeneralpublic
Whataretheyasking?
• Topicsconsistentregardlessofcollec5on/site– Didyouknowthiswasbroken/wrong?– NowthatI’vefoundthis,howcanIgetaccess?– CanIusethisimageinmybook/project?– Ihavethisoldbook–isitvaluable?– Canyouhelpmewiththisverycomplexquery?
• ThepercentagesvaryinHathiTrust– Moreaboutcontenterrorsandaccessdesires
WhataretheynotaskingaboutHathiTrust?
• Ques5onsrelatedtothecontent• Manyrepeatedques5onsinsomeofourcollec5ons– WhatisApocrypha?WhyisEcclesiates9showingasQoh.9?Whyisn’t
thisquoteinyouronlineCollectedWorksofAbrahamLincoln?Whycan’tIfindtheverse9:11aboutthewrathoftheeagleinyouronlineKoran?
• Alsogeneralques5ons– Thiscensusfrom1901listspeopleborninthistown,butnoneofthe
gazeQeersoratlasesshowatownofthisname.ThePhiladelphiaCentennialexhibiFonhistorysaysthis,asouvenirmedallionfromarelaFvesayssomethingelse.Whichisright?Isitfake?AreyousurethesepoemsinCatholicWorldarebyWilliamGibson?
Observa5ons
• HathiTrustfeedbackformsguideyoutoerror/accessrepor5ng,notsubjectinquiries
• Single5tlesites(Bibles,Koran,Lincoln)getthemostrepeatedcontentques5ons
• MoAgetsmanycontentques5ons,buthasaunifying(ifgeneral)theme
• Affilia5onwithMLibrarymaymakecontentques5onsseemmorewelcome
• Dataskewedbysomuchpastprac5ce
HathiTrustUserSupport
JeremyYork
Timeline
• October2008–HathiTrustlaunched– hathitrust‐[email protected]
• February2010–star5ngregularsta5s5cs• August2010–Chrisjoinedhathitrust‐info• UsersupportgroupsinceApril2011
WhatDigitalPublicServiceAccomplishes(1)
• Howdoyoufindoutwhoyourusersare,andwhattheymostneed?– Userfeedbackviathefeedbacklink– Frequentdiscussionswithreference&instruc5onlibrarianswhosharetheir"front‐line"encounterswithusers
– HathiTrustUXAdvisoryGroup&UX‐SIGforinterestedstaffatpartnerins5tu5onstodiscussandbringissuestothetable• Developingsetofpersonasandscenarios
– Watchblogs&twiQerforunsolicitedfeedback– Somewhatregularuserresearch(surveys)
Usesta5s5cs
Visits %newvisits Pageviews Pages/visit Timeonsite
Jan‐Jun2010 253,129 69% 2,154,385 11.6 6.3min
Jul‐Dec2010 907,524 75% 8,443,692 9.3 5.3min
Jan‐Jun2011 2,154,385 83% 14,945,119 6.9 4min
Jul‐Oct2010 2,274,468 84% 12,072,991 5.3 3.3min
WhatDigitalPublicServiceAccomplishes(1)
• What’sthemostusefulthinguserscandoforyou,andhowcanyoubestgetthemtodoit?– Letusknowofproblems,issues,desires(contentandservices)
– Peoplewhoareinterestedletusknow
UserSupportIssues
0
50
100
150
200
250
300
350
400
450
Apr‐Jun2011 Jul‐11 Aug‐11 Sep‐11
Content
Cataloging
AccessandUse
WebApplica5ons
PartnerIngest
General
IssueType AugustIssues SeptemberIssues
Content 110 171Quality 96 154Non‐partnerDigitalDeposit 3 2Collections 8 4
Cataloging 26 25AccessandUse 111 127
Copyright 58 73Permissions 23 12Takedown 2 3PrintonDemand 6 17Inter‐libraryloan 0 5Full‐PDFore‐copyrequests 14 24Datasets 1 1DataAvailabilityandAPIs 1 7Reuseofcontent 7 5
Webapplications 27 22Functionalityproblems 5 5ProblemswithloginspeciNically
1 0
GeneralQuestionsaboutlogin 3 2Partnerssettinguplogin 4 5Usabilityissues 11 6Featurerequests 7 2
PartnerIngest 2 0General 59 65
Partnership 13 12Infrastructure 1 0Miscellaneous 45 53
hQp://www.hathitrust.org/wg_user_support_issue_types
WhatDigitalPublicServiceAccomplishes(2)
• Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?– Ensurethatwelookatmanydifferentsourcestogetthelargestvarietypossible.
– Wereceive,find,andsolicitfeedbackinwaysnotedabove
WhatDigitalPublicServiceAccomplishes(3)
• Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?– WearepreQyuniform
Implemen5ngDigitalPublicServices(1)
• Doyouhaveservicegoalsandbenchmarks?Howdoyouevaluatethem?– Respondtofeedbackwithin1businessday– Handleissuessuchastake‐downno5cesandcri5calsystemsproblemsimmediately
– Nogoalsforresolvingotherproblems,butcorrec5onshappenpreQyquickly
– Usersupportasoutreachmechanism• Importanttohowweareseenbythecommunity
Implemen5ngDigitalPublicServices(2)
• Howdoyouallocatestaff,anddotriage?– Usersupportgroup– Rota5onof24‐hour“on‐call”periods
Implemen5ngDigitalPublicServices(3)
• Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?
Users(partnerandnon)
JIRA
UserSupportWorkingGroup
UniversityofMichiganContacts• Copyright• Quality• PrintonDemand
AllHathiTrustContacts
Documenta5on
Implemen5ngDigitalPublicServices(4)
• Haveyouchangedyourdesignorwhatyoudobasedonuserfeedback?How?– PageTurnerimprovements• BookReaderintegra5onandaccompanyinginterfacereorganiza5on;full‐screenmechanism• Advancedsearchfeaturesforfull‐textsearch• LabelingforPDFdownload
Evalua5ngandScalingUpDigitalPublicService(1)
• Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdoyoufindmosteffec5ve?– No,butkeyfactorinexpandingtoworkinggroupandhavingpartnersystemwascommunica5on
• Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?– Commonly‐usablesystem– Definedroles(contacts,etc.)– Definedworkflowsandprocedures
• Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?– Factoritin,don’tunderes5mate5meneeded
Evalua5ngandScalingUpDigitalPublicService(2)
LibraryofCongressDigitalCollec5ons
LeslieJohnston
SoMuchStuff,It’sUsedasaUnitofMeasure
• Whatcons5tutesa“LibraryofCongress”worthofdigitalcontentchangesallthe5me.
• Ahugevarietyofformats:FullText,PageImages,ImageCollec5ons,FindingAids,ElectronicSerials,Video,Audio,Legisla5on,WebArchives.– Anes5mateof24.6millionfilesinthemainLCwebpresenceattheendof2010.
• Avarietyofacquisi5onmethods,includingthroughtheUSCopyrightOffice
• Avarietyofaccessrules(includedclassified)andaccessmethodsandsystems.
Ques5onsareAlwaysComingIn
• Viaemail• Via“AskaLibrarian”OnlineReference• InPerson• In2010,Librarystaffansweredover191,000emailoronlinereferenceques5ons.
Whoareweserving?
• Congress• Librariansatotherins5tu5ons• AcademicResearchers
• TheGeneralPublic
WhatDoTheyWanttoKnow?
• DoyouhavethisthingthatIcouldn’tfind?Youmusthaveit,becausetheLibraryofCongresshaseverything.
• Whyisn’teverythingfulltext?• HowdoIgettherightstouseyourcollec5ons,andget
permissionstousetheminmyresearchorinapublica5on?• HowcanIgetcopiesofyourdigitalfiles?• Willyoudigi5zesomethinginyourcollec5onsforme?• HowdoIciteyouronlinecollec5ons?• Pleasefixthiserrorinyourwebsite,orinyour
bibliographicorauthorityrecords.• CanIdonatedigitalcollec5onstotheLibrary?• WilltheLibraryhelpmedigi5zemycollec5on?• Whatstandardsdoyouusefordigi5zing?
WhatDigitalPublicServiceAccomplishes
• Howwelearnsomethingaboutwhooutusersare,andwhattheyneedmost:– Interac5onswithDigitalReferencelibrarians– UserfeedbackviatheContactForm– CommentsontheLibrary’sblogs– ShareToolandWeblogsta5s5cs• Over580millionpage‐viewsoftheLibrary’swebsitein2010.
WhatDigitalPublicServiceAccomplishes
• Whatwelearn:– Whatarewedoingwrong?OneissueisthecomplexityoftheLibrary’swebpresence.Wenowhaveanini5a5vetomakeiteasiertofinddigitalcollec5onswithoutknowingabouttheminthefirstplace.
– Whatarewedoingright?Wegetmoreposi5vereac5onsandthanksfordoingwhatwedothannega5ve.
WhatDigitalPublicServiceAccomplishes
• Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?– Wereadthemessagesthatweget.– Weholdourselvestoaveryhighstandardofrespondingtoallreferenceinquiries.
WhatDigitalPublicServiceAccomplishes
• Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?– OurfeedbackfromCongressisdifferentfromourfeedbackfromthepublic• Congressmakesdirectdigitalcollec5onbuildingandservicesdependingonthetasksathand• Researchersandthepublicarerela5velyuniform
Implemen5ngDigitalPublicServices
• Doyouhaveservicegoalsandbenchmarks?Howdoyouevaluatethem?– Respondtoques5onsandfeedbackwithin5businessdays
– 4‐6week5meframefordigitalreproduc5onrequests
– Nobenchmarksforresolvingotherissues
Implemen5ngDigitalPublicServices
• Howdoyouallocatestaff,anddotriage?– Variesbycustodialunitandcollec5on– DigitalReferenceTeamshaspostedchathours,andarota5ngdutyroster
Implemen5ngDigitalPublicServices
• Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?– OnlineChat– Contact/FeedbackForms– OnlineFAQs– Documentedonlineguidelinesforuseofthecollec5ons
– Documentedstandardsonline
Implemen5ngDigitalPublicServices
• Haveyouchangedyourdesignorwhatyoudobasedonuserfeedback?How?– Ofpublicservices?Notreally.– Ofouroverallwebpresence?Thatishappeningnow,tohelpusersbeQerdiscoverourcollec5ons.
Evalua5ngandScalingUpDigitalPublicService
• Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdoyoufindmosteffec5ve?– No,wedonot.Butthatdoesn’tmeanthatwedon’treportonthem.
• Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?– Sharemoreinforma5onaboutwhatcollec5onsanddigi5zedcollec5onswehave• Explicitlydocumentrightsformediafilesandmetadata• Makethemavailableaslinkedopendatawherewecan
– Common‐ishstandards,oratleastwell‐documentedstandards
– Consistent,documentedworkflows• Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?– Thescalereallydoesmakeadifference.Ourresponse5mesarealwaysgoingtobeslower.
Evalua5ngandScalingUpDigitalPublicService
TheOnlineBooksPage
JohnMarkOckerbloom
• 1personworkingpart‐5mefallsbehind– Bothincatalog,andinrespondingtopublic
• Scalingupthecatalog– Metadataautoma5callydownloadedfromHathiTrust,othersources
– Icon5nuetoaddnewentries,“curate”auto‐loadedentries,onmyownandatuserrequest
• Scalingupuserservice– Invitepeopletostandardforms– Makeefficientbackendtodealwithrequests
• Ul5mately,nosubs5tuteforhumaninvestment
Tes5ngthelimitsofalow‐resourcecatalogproject,since1993
Feedbacklinksinresults
Requestcura5on
Userstellmewhattheywant
Andcanhearsomethingback
WhatIseeonmyend
EnsuringIgettoeveryone
• Numberofonlinebookss5lltogrowlots– EspeciallyasIaddmoreautoma5callyloadedsources
• Staff,budgetnotlikelytogrow– Userfeedbackhelpsmedeterminewhereeffortbestspent
• Enhancedback‐endinterfacemaymakemul5plemaintainersfeasible
• Maybeusefulcollabora5onswithinterns,volunteers– Awaytogethands‐onexperiencewithlibrarianship
• Sharingdatahelpsotherservicesbuildonmywork– Enhancing“regular”librarycollec5onsmayenablesupport
Lookingahead:Collabora5on
Discuss:Goalsofdigitalpublicservice
• Howdoyoufindoutwhoyourusersare,andwhattheymostneed?
• What’sthemostusefulthinguserscandoforyou,andhowcanyoubestgetthemtodoit?
• Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?
• Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?
Discuss:Implemen5ngdigitalpublicservice
• Doyouhaveservicequalitygoalsandbenchmarks?Howdoyouevaluatethem?
• Whodoesthework?Howdoyouallocatelabor,anddotriage?
• Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?
• Hasuserfeedbackpromptedyoutochangeyourdesignorwhatyoudo?How?
Discuss:Evalua5ngandscalingupdigitalpublicservice
• Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdatadoyoufindmosteffec5ve?
• Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?
• Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?– Whathaveyoulearnedthatyoudidn’texpect?
Collabora5on
PhotobyMaryMarkOckerbloom
