“But What Does It Mean?” Using Statistical Data for Decision

Making in Academic Libraries

Steve Hiller

University of Washington Libraries

Seattle, Washington USA

North American Academic Libraries

• Swimming (or is it drowning?) in data– Electronic resources/collections use data– User assessment/survey data– Performance measures/outcomes based data

• ARL New Measures Initiative

• User-Centered Libraries/Culture of Assessment (organizational refocus)

Assessment Needs DLF report 2002

• Collect meaningful, purposeful data

• Develop skills to gather, analyze, interpret, present and use data

• Develop comprehensive assessment plans

• Organize assessment as a core activity

• Compile and management assessment data

• Understand trends

DLF Report – Bleak Outlook

“The results of the DLF study suggest that individually, libraries in many cases are collecting data without having the will, organizational capacity, or interest to interpret and use the data effectively in library planning.”

Denise Troll Covey (Author)

But Wait! There is Optimism!

• Value of data recognized• Data definition and collection more standardized• Performance measures better articulated• National impetus in higher education and libraries• Organizational structure change;strategic planning• MIS/Assessment positions growing in libraries• Success stories growing

Obstacles to Using Statistical Data

• Too much data• Not the right type of data• Timeliness• Library organizational structure• Statistical skills and understanding• Innumeracy

STATISTICAL LITERACY?

“Everyone knows that you can use statistics to prove anything.

14% of all people know that.”

Homer Simpson

The Data

Think critically:• How and why were they generated?• Where do the numbers come from?• What do they represent?• Can you compare them? To what?• Do they make sense?• Can we use data to improve performance?

Communicating Results Keep it Simple: Less is More

Provide executive summary or equivalent

Briefly cover scope and methodology

Identify key findings (not all results)

Mix text, data, and graphics

Avoid jargon

Make it understandable to your audience(s)

Identify action items or follow-up

Presentation for Management

• Presentation is different for management than for research publication. Highlight:– What’s important– How the information can be used to improve

services– Ways to communicate externally and internally– Recommendations for handoffs

Be Graphic!!!

“Often the most effective way to describe, explore and summarize a set of numbers – even a very large set – is to look at pictures of those numbers. Furthermore, of all methods for analyzing and communicating statistical information, well designed data graphics are usually the simplest and at the same time the most powerful.”

Edwin Tufte

The Visual Display of Quantitative Information

                         

                                 

          

                          

                                 

Edward Tufte"The Leonardo da Vinci of data." THE NEW YORK TIMES

• The Visual Display of Quantitative Information (new second edition)

• Envisioning Information

• Visual Explanations

Edward Tufte"The Leonardo da Vinci of data." THE NEW YORK TIMES

University of Washington Libraries

Assessment • Large scale user surveys done triennially since 1992• In-library use surveys done triennially since 1993• Annual topic for focus group sessions since 1998• Observation and usability studies on computer use• E-metrics in online usage• Process improvement in technical services

UW Libraries Assessment Communication

• Within 2-4 weeks after data receipt:– Initial data and analysis posted internally– Presentations made to library management

• Within 4-8 weeks after getting results:– Data and analysis posted on public Web site

http://www.lib.washington.edu/surveys/

– Presentations made to other library groups

• After 8 weeks: – Written reports for local publication– Presentations made to external groups

UW Libraries Assessment Web Sitehttp://www.lib.washington.edu/surveys/

Faculty and Student Surveys

Library Directions Article

Triennial Survey – Response Rate

2001 Sent

2001 Returned

2001 1998 1995 1992

Faculty 3720 1345 36% 40% 31% 28%

Grad 1500 597 40% 46% 41% 56%

Undergrad (2002)

2000 497 25% 39% 23% 41%

LIBRARY “DISSATISFACTION” IS LOWPercent Not Satisfied (Those marking 1 or 2 on scale of 1 to 5)

Frequency of In-Person Visits and Remote Access Uses 1998/2001 (% who use at least weekly)

Faculty Library Use Pattern 2001 by Academic Area(Those who use libraries at least weekly)

5.20% 4.20% 3.90%

30.80%39.50%

53.50%

51.10% 37.60%24.20%

0.70%2.30% 2.70%

12.40% 16.40% 16.80%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Humanities-Soc Sci Science Health Science

Visit Only Remote Only Both Other Don't Use Weekly

UWAnnual Loans and In-Library Use 1995 to 2001

ARL “Total Circulation” 1995-2001 (Mean/Median)

Using Statistical Data in ManagementNot!

“Many DLF respondents reported surveys whose results were never analyzed . . . Libraries appear to be slow in acquiring the skills needed to use survey data.” (Troll, 2002)

Barriers to using LibQUAL+ data by participants: “A dearth of in-house statistical skills, a lack of organizational culture that encourages assessment, low sample sizes, lack of time and money to work with results, need for more documentation to accompany the data.” (Waller and Hipps, 2002)

Using Statistical Data in ManagementYes!

• Library uses data in decision making• Staff have data analysis skills• Data is timely• Resources are available to effect change• Communicate results/action in timely manner• Validate with complementary information• Organization primed for action

University of Washington Libraries Data Use and Action

• Facilities planning and operation

• Open hours

• Access to online information

• Service improvements

• Recognizing a diverse user community

Reasons for Visiting Libraries 2001(Among those visiting at least weekly)

Facilities Action

• Libraries are student places

• Provide diversified study and work areas

• Add computers (both “library” and those with application software)

• Extend hours of opening (but not reference)

• Increase user space; decrease collections space

User Priorities for Library are Online AccessTop Priorities Identified in 2001

Library Priority Faculty Graduate

Students

Undergraduates

Full-text to desktop

73.5% 72.8% 54.9%

Online access to older journals

59.6% 61.8% 40.0%

Maintain quality of print collection

57.4% 53.3% 24.5%

Electronic reserves 23.8% 37.5% 49.5%

Priorities Differ by Academic Area 2001

Electronic Information Action

• Cancel print subscriptions if electronic available in sciences and health sciences

• New subscriptions are electronic only, if available• Consortial purchases for more ejournal content• Purchase electronic backfiles for older journals• Provide access to full-text aggregator products• Purchase online editions of reference works

Conclusion

• Use of Statistical Data in Library Management is Growing in North American Research Libraries

• Further progress needed:– Standardized data definitions and collections methods

– Library staff expertise in data analysis and presentation

– Organizational culture that facilitates change

– Institutional planning and evaluation process that establishes goals and priorities that are measurable