Post on 17-Jan-2016
description
transcript
Evaluating Electronic Resources
Charleston Conference 2005
Evaluating Electronic Resources:A Bird’s Eye View
Welcome & Introductions Audrey Powers
Evaluating Electronic Resources: A Cooperative Process with Objective Results
Break George Machovec
Consortial purchasing Gold Rush (ERM and content analysis) The Charleston Advisor & Snapshot Reviews
Q & A
Evaluating Electronic ResourcesA Collaborative Process with Objective Results
Evaluate electronic resources for acquisition, retention and withdrawal purposes
Identify essential criteria Customize the evaluation form Administer the evaluation process manually or
electronically
If you use this process or these slides, please ask for permission and give me credit.
Thank you.
Audrey PowersResearch and CollectionsUniversity of South Florida
4202 E. Fowler Avenue, LIB122Tampa, FL 33620
813-974-9001(Phone)813-974-9875 (Fax)
apowers@lib.usf.edu
This evaluation process was developed to
Guide evaluators
Collaborative processObjective results
Enables institutional evaluations
Acquisitions Retention
WithdrawalComparisons
Advantages
Institutional objectives are metAdaptable for a variety of user populations
Electronically administeredCustomizable
Evaluate Electronic Resources
Databases eJournals eBooks Open Access resources Internet sites Hardware Software
Courseware
Evaluators Opportunity to query a diversity of users
StudentsFacultyAdministratorsLibrariansStaff
Institutions Variety of institutions
Libraries School Public Academic Special
Schools BusinessesThe Charleston Advisor
Considerations Purpose Type and size of institution Extent of existing resources Audience Observed use patterns Desired results Project timeline Budget allocations
Used for
DatabasesEntire collection/Discipline specific databases
Trial databasesSingle trial/multiple trials
Database comparisonsTrial database/existing database
Why do it?
Cut in funding Rising costs Duplication Librarians have “favorites” Some databases not used
Why do it?
110 databases 80 databases are free or reduced cost due to
consortial arrangements $65,400.00 20% of the operating budget
University of Montevallo
Why do it?
512 databases 200 databases are free or reduced cost
due to consortial arrangements $852,680.00 17% of the operating budget
University of South Florida
Why do it?
McCracken, Peter. "A comparison of print and electronic journal holdings in academic and public libraries." Libri 53.4 (2003): 237-241.
Institution type Avg # of
print journals
Avg # of
e-journals
% of
e-journals
Associate(n=60)
874 5,617 86.5%
BA(n=60)
1,372 6,841 83.3%
Masters(n=60)
3,178 7,910 71.3%
Doctoral(n=60)
5, 046 9,836 39.5%
Lg. Public(n=60)
3,529 6,362 64.3%
USF - 2003
USF - 2005
4,584
4,225
17,008
20,022
78.8%
82.6%
Expenditures
Journals $1,528,580.00 Databases 860,556.00 Books 152,920.00 CD-ROMs 2,206.00 Total $2,544,262.00
University of South Florida
Three Step Process
Prepare
Gather Data
Results
Phase I
PREPARE
Identify Known Data
Cost (actual cost per year) Use (annual use statistics) Cost per use Duplication (duplication of journal titles) Peer comparisons (availability at peer
institutions)
Develop Evaluations
Content Unique content Ease of use Instruction Overall quality Need Comments
ASK
QUESTIONS
?
Content
Is the content and reading level appropriate for the intended audience?
Is the coverage selected or comprehensive?
Is the content substantial? Is this the only resource that covers this
topic?
Unique Content
Are the journal titles unique to the database?
Is the search interface common or unique? What are its special features?
Ease of use
Is the format and layout easy to navigate? Is a standard search methodology in use? What retrieval methods are available? Is access to the database and search
screen easy?
Instruction
Is the database easy to teach? Is the number of simultaneous users
limiting? Does the licensing agreement restrict
class instruction? Is it useful for assignments?
Overall quality
Are there many errors? Is the database often inaccessible? Is the documentation useful, well written
and easy to navigate? Are use statistics available?
Need
What is the value to the institution? Is it required? Does it fulfill a need?
Comments
Gives the evaluators an opportunity to express opinions
DATABASE EVALUATION FORM
Database ___________________________________________________________
URL ___________________________________________________________
Evaluator___________________________________________________________
Known DataCost:
Use:
Cost/use:
Duplication:
Peer comparisons:
Evaluations
Content: 0 1 2 3 4
Unique Content: 0 1 2 3 4
Ease of use: 0 1 2 3 4
Instruction: 0 1 2 3 4
Overall quality: 0 1 2 3 4
0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory
Need: ____Inappropriate _____Useful ____Essential
Comments:
FACULTY EVALUATION FORMPlease complete and submit this form.
Access the electronic resource by clicking on the hyperlink.
Electronic Resource
Known DataCost:
Use:
Cost/use:
Peer comparisons:
Evaluations
Content: 0 1 2 3 4
Unique Content 0 1 2 3 4
Ease of use: 0 1 2 3 4
Instruction: 0 1 2 3 4
Overall quality: 0 1 2 3 4
0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory
Need: ____Inappropriate _____Useful ____Essential
Comments:
STUDENT EVALUATION FORM Please complete and submit this form.
Access the electronic resource by clicking on the hyperlink.
Electronic Resource
Content: 0 1 2 3 4
Unique Content: 0 1 2 3 4
Ease of use: 0 1 2 3 4
Assignment use: 0 1 2 3 4
Overall quality: 0 1 2 3 4
0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory
Need: ____Inappropriate _____Useful ____Essential
Comments:
Phase II
GATHER DATA
Gather essential data
Obtain Known Data Distribute evaluation form(s) Conduct Evaluations Compile data
Phase III
RESULTS
Report Results Collate results Present results Recommend an action plan
In Summary Prepare
Determine criteria Develop evaluation form
Create web forms e Create a database e
Distribute/publish Gather Data
Distribute evaluation forms Email evaluation forms/Publish on web page e
Conduct evaluations Evaluations are completed online e
Compile results Compile the results in Access > Export to Excel e
Results Collate the results
Manipulate the data in Excel e Determine the average for each criterion e Determine the composite score e
Present the results Recommend an action plan
eBenefits
Customization Distribution Participation Response rate Collation of data
The eProcess
The eProcess: An Overview
Create web forms Create a database Compile results in Access Export results to Excel
Manipulate data in ExcelDetermine average for each criterionDetermine composite score
HOW TO DO IT
Create Web Forms
Create a new page
Create web formsPlace cursor in the body of the page
Go to Insert > Form > Form
Hit Enter 2x
Create Web Forms
Go to Insert > Form > Option Button, etc.
Create web forms
Click 2x on Option Button and complete Group name: and Value: fields
Create web forms
Create web forms
Create a confirmation page
Create web forms
Preview confirmation page in Browser
Copy URL of confirmation page
Create databaseRight click in the body of the form
Select Form Properties
Create a database
Select Send to database
Under Form Properties complete Form name:
Create a database
Select Options > paste URL of confirmation page (optional):
Create a database
Click on Create Database…
Create a databaseClick on Add Connection…
Select name of database
Click on Verify
Click Apply > OK > OK > OK
Create a database
Go to File > Save As
Change database extension to .asp and Save
Create a database
Create a database
Check the fpdb folder for the database
Create a database
Test form and confirmation page in Browser
Create a database
Compile the results in Access
Review the results in Access before the page is published:In FrontPage, go to fpdb file and click on name.mdb > Open > Results
Compile the results in Access Review results after the page is published:
Go to IE, open http://website/fpdb/nameoffile.mdb
Compile the results in Access
Export results to Excel
Example
Determine averages & Composite score
Final Results
N=27 Respondents included faculty, staff, students, librarians
Content 3
Unique content 3
Ease of use 3.11
Instructional value 3
Overall quality 3.19
Composite score 3.06
Comments
Great database. We need this for our collection. What is this? This is ridiculous. Are you serious? Great database. Take it or leave it. Good for knowing simple facts, but gives little background information. It seems as though the database may duplicate some of the other literature database we already
have, but it seems useful that reference materials are collected in the same place. Very useful. Extensive degree of information. Also very fun! Reference mapper is a neat feature. I used this for my class. I plan to use it from home in the future. I found the site very difficult to use and very limited in what it gave in return. Good. VERY HELPFUL! Definitely needed! It seems to be mostly a lot of dictionaries, some of the dictionaries go in depth on a topic, which
is nice.
MORE
RESULTS
RESULTS
Science Databases
Cost Use Cost/Use Duplication (≥50%)
Peer Comparisons
Evaluations (0-4)
Need
ASTI .62 1 .62 OK 5 of 17 2.6 1
BasicBIOSIS 22.32 36 .62 69.8% 11 of 17 3.2 3
BioAgIndex 22.32 36 .62 50% 7 of 17 2.6 2
EventLine 0 0 .62 NCD 0 of 17 2.3 1
General Science
25.42 41 .62 52% 15 of 17 3.2 3
Show results from the Art databases or xreferplus
FORMS
USED
DATABASE EVALUATION FORM
Database ___________________________________________________________
URL ___________________________________________________________
Evaluator___________________________________________________________
Known DataCost:
Use:
Cost/use:
Duplication:
Peer comparisons:
Evaluations
Content: 0 1 2 3 4
Unique Content: 0 1 2 3 4
Ease of use: 0 1 2 3 4
Instruction: 0 1 2 3 4
Overall quality: 0 1 2 3 4
0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory
Need: ____Inappropriate _____Useful ____Essential
Comments:
FACULTY EVALUATION FORMPlease complete and submit this form.
Access the electronic resource by clicking on the hyperlink.
Electronic Resource
Known DataCost:
Use:
Cost/use:
Peer comparisons:
Evaluations
Content: 0 1 2 3 4
Unique Content 0 1 2 3 4
Ease of use: 0 1 2 3 4
Instruction: 0 1 2 3 4
Overall quality: 0 1 2 3 4
0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory
Need: ____Inappropriate _____Useful ____Essential
Comments:
STUDENT EVALUATION FORM Please complete and submit this form.
Access the electronic resource by clicking on the hyperlink.
Electronic Resource
Content: 0 1 2 3 4
Unique Content: 0 1 2 3 4
Ease of use: 0 1 2 3 4
Instruction: 0 1 2 3 4
Overall quality: 0 1 2 3 4
0=Inappropriate 1=Very Unsatisfactory 2=Unsatisfactory 3=Satisfactory 4=Very Satisfactory
Need: ____Inappropriate _____Useful ____Essential
Comments:
DATABASE GROUPINGS
Team Leader 1
Due Date
Subject
Team Leader 2Due Date
Subject
Team Leader 3Due Date
Subject
Team Leader 4Due Date
Subject
Team Leader 5Due Date
Subject
Database 1 Database 1 Database 1 Database 1 Database 1
Database 2 Database 2 Database 2 Database 2 Database 2
Database 3 Database 3 Database 3 Database 3 Database 3
Database 4 Database 4 Database 4 Database 4 Database 4
COST PER USE ANALYSIS
Database Name
Cost Use Cost/Use Comments
Database 1 LC LU LC/LU
Database 2 HC HU HC/HU
Database 3 LC HU LC/HU
Database 4 HC LU HC/LU
DATA SUMMARY
DatabaseName
Cost Use Cost/Use Duplication (≥50%)
Peer Comparisons
Evaluations (0-4)
Need
The Charleston Advisor
Composite score Content Searchability Price Contract
Audrey PowersResearch and Collection Development Librarian
University of South Floridaapowers@lib.usf.edu
Handbook of Electronic and Digital Acquisitions, 2006|“Evaluating Databases for Acquisitions and Collection Development”