Application of EZProxy logs, Voyager’s Patron Database, MySQL, and

ColdFusion for Comparative Usage Analysis of Library Resources

Ray Schwartz, William Paterson University of NJ,

schwartzr2@wpunj.eduENUG Conference, University of Bridgeport, CT

Thursday, October 27, 2011

2

Our University

• 9000 undergraduates• 1000 graduates (mostly education majors)• 400 faculty• 800 adjuncts• 1000 staff

3

Our Library

• 19 librarians and 26 library staff• 350,000 volumes• 18,000 audiovisual items• 47,000 print and electronic periodicals • 124 general and subject specific databases• $1,100,000 Non-Salary Allocations

EZProxy via LDAP authenticates access to:

• Databases• Electronic journals• ILL/Doc Delivery forms

5

Example of EZProxy log entrynj.dhcp.embarqhsd.net

-

theuser

1/1/2008 4:25:15 AM

GET

http://ezproxy.wpunj.edu:2048/connect?session=sGHMbeSss121YxZa&url=http://www.wpunj.edu/scripts/webscript.exe?fs.scr

HTTP/1.1

302

537

http://ezproxy.wpunj.edu:2048/login?url=http://www.wpunj.edu/scripts/webscript.exe?fs.scr

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)

• Ip address

• (Not used)

• user id

• date/time

• Method

• page retrieved

• Version

• response code

• no. of bytes

• Referring URL

• User agent

• EZProxy transactions would be stored in one table with patron statistical categories, but without the user ID.

• User ID s would be stored in another table with counts for each service divided by academic year.

• Logs are collected monthly and loaded and deleted monthly.

6

MySQL operations

7

Perl Script for loading ezproxy log into MySQL

use strict;my %month=(Jan=>'01',Feb=>'02',Mar=>'03',Apr=>'04',May=>'05',Jun=>'06',Jul=>'07',Aug=>'08',Sep=>'09',Oct=>'10',Nov=>'11',Dec=>'12');while (<>){ my $pattern = '^(\S*) (\S*) (\S*) (\S*) '. '\[(..)\/(...)\/(....):(..):(..):(..) .....\]'. ' "(\S*) (\S*) (\S*)" '. '(\d*) (-|\d*) "([^"]*)" "([^"]*)"'; if (m/$pattern/){ my ($tgt,$ref,$agt) = (esc($12),esc($16),esc($17)); my $byt = $15 eq '_'?'NULL':$15; print "INSERT INTO ezproxylogs VALUES ('$1','$2','$3',". " TIMESTAMP '$7/$month{$6}/$5 $8:$9:$10','$11','$tgt',". "'$13',$14,$byt,'$ref','$agt');\r."; }else{ print "--Skipped line $.\n"; }}sub esc{ my ($p) = @_; $p =~ s/'/''/g; return $p;}

8

• Voyager Patron Database allows a maximum of 10 statistical categories per patron record.

• Weekly extract from SIS and HRS to load into

Voyager

Patron Statistical Categories

•College and Mercer Identifier•Class Level (Freshman, Sophomore, Junior, Senior, Graduate)•Total Hours Registered for Current Semester•Major•2nd Major•Degree•CA-Collection Agency•SOILS•Student Entrance Level (New Non-Traditional Freshman, New First Time Transfer, etc.)•Department

From Students

From Faculty / Staff / Adjuncts

•College•Full or Part-Time•Status (Faculty, Adjunct, Staff, Professional Staff, Tenured, Tenure-Track)•Division•Departments

Extracting patron statistical categories back out of Voyager and building them into MySQL database.

Once completed, user ids are deleted.

Internal WPUNet IPv4 addressing scheme IP Network 149.151.

Block Description IP Block Start Range IP Block End Range

Admin (Services) 2.1 129.254Labs (Users) 130.1 162.254Admin (Users) 163.1 233.254Resnet (Users) 234.1 250.254Video 251.1 251.253

Admin VLANs Labs VLANs

Vlan ID Vlan Name Vlan ID Vlan Name

2 Servers 3 Lab Servers

4 Admin 9 Imaging

5 Science 160 Lib Labs

6 Test Servers 174 STU VPN

7 NAS 175 Ben Shahn Lab

101 Energy Management 178 Hobart Lab

102 Diebold 179 SCI Lab

104 Xerox 187 CS Lab

150 Media Services 192 Atrium

161 Dorms Offices 209 Labs

162 RBI 212 Resnet Labs

163 Police 214 Raub Labs

164 Maintenance 228 VR Labs

IP Address Location = 149.151.VlanID.*

14

References

•Coombs, Karen A. (2005). Lessons learned from analyzing library database usage data. Library Hi Tech, 23:4, 598.• Diana, Birkin James. dashboard_beta. http://library.brown.edu/dashboard/info/• Metridoc. http://code.google.com/p/metridoc/• Morton-Owens, Emily (2011) Trends at a glance. LITA 2011. http://connect.ala.org/files/79651/trends_at_a_glance_dashboards_pdf_12068.pdf

25

Questions?

Ray Schwartz, Systems Specialist Librarian

Cheng Library, William Paterson University, Wayne, New Jersey, USAschwartzr2 @ wpunj.edu