Brenda MatheniaAssistant Professor/Reference Librarian

Montana State University - Bozeman

MPLA/NLA 2003Incline Village, NV

Library Instruction Assessment in Real Life

Instruction Assessment efforts

at The Libraries

Montana State University - Bozeman

Why Assessment

Student success Improving learning Improving teaching Support tenure application

A Library InstructionAssessment Plan

January 2002-July 2002 authored in-house document on library instruction assessment

July 2002 presented to faculty & admin

Skepticism from faculty

My next step - lead by example

The Reluctant Professor(s)

Instruction program poorly formed no instruction plan

no goals, no objectives, no common approach support for my efforts - “sort of” perceptions needed modification

The Nutty Professor

The Choice: keep “planning” or “just do it”?

The decision: “Just do it!”

So I stepped to the high-dive and jumped in!

Just do it!

“Set your sights lower and do the best evaluation you can with what you have.” (Barclay 1993, p 196)

Donald Barclay. “Evaluating Library Instructionn: Doing the Best You Can with What You Have,” RQ 33 (Winter 1993): 195-202.

Becoming a Guinea Pig

I did the draft assessment plan I would do the first assessment Helping others find comfort level More importantly, helping me find the way

through the quagmire of assessment.

My Opportunity

Seeking opportunity no semester long credit courses needed a class with more than single contact Preferably one I was familiar with already

Needed sympathetic department faculty First one available - Biochemistry 204N, a

biochemistry course for non-majors

Biochemistry (BCHM) 204

Mostly lower level undergrads Most new to library research Complex concepts found within this class Faculty member supportive of library and

information literacy Faculty member a challenge to work with

The Goal

To Learn! To Learn! To Learn!

The Approach

Specific Outcomes Pre-Test Instructional tools - worksheets Post-test Evaluation of Bibliographies

Student Outcomes: 1-3 of 6

1. Students can identify appropriate information sources.

2. Students can develop successful search strategies to find information in a variety of resources

3. Students can locate material in the library

Student Outcomes: 4 & 5 of 6

4. Students can evaluate the relevance of retrieved information

5. Students use information and information resources appropriately to produce quality research in final paper/presentation.

The Final Outcome: 6 of 6

#6 Students appreciate the value of information (library) skills for life-long learning

This is a higher order expectation….

Realized after the first class that this was beyond my ability to teach or evaluate. Maybe at a later stage in my own assessment skills development!

The Process

Developed pre/post-test with questions geared to the outcomes

Designed specific classroom worksheets to focus our time in class on skills Topic Identification and Search Strategies Library Catalog and Citation Exercise Journal Article Research Exercise

The Worksheets: #1

Topic ID and Search Strategies To help break topic into searchable terms and

understand the process of creating a search strategy for specific databases.

Ideas surrounding synonyms, broader/narrower topics. Begin to understand the iterative process involved in

most research.

The Worksheets: #2 Library Catalog & Citation Exercise

To familiarize with online catalog, finding a known book and locating the call number of a specific journal

Identifying key pieces of information in a catalog record – how to read the catalog

How to decipher an article citation from a full-text database

How to proceed with locating a journal article with only a citation.

The Worksheets: #3

Journal Article Research Exercise Searching specific database (PubMed) for a given term Finding references to articles with known name Helping them navigate a database that provides some

full text but not always……and what to do about it How to leap from the indexes to the catalog to track

down material

The Test(s)

Pre-test given a week before their first information lab By me first thing during a lecture All but 3 were picked up, 3 dropped off after class and

2 did the pre-test before information lab for 19 tests Post-test given 2 weeks before end of classes

By classroom professor Only 15 post-tests received

The Results

Out of 15 students who took both the pre and post-test, 9 improved their scores

Two (2) maintained same score Four (4) had scores that decreased

disinterest frustration of end of term Professor administered, I did not (problem)

Average score increased 2.2 pts. (23.5 to 25.7)

What I Learned

I learned about tests will use more extensive test next time, including

short/long answer type I think they do (or can) help us judge learning

Carefully choose faculty to work with need more contact with students more involvement in entire course syllabus

The Next Steps

Find larger group to work with, more extensively (General Studies?, English 121?)

Build a new test, include plan for statistical analysis (in-house or through institution)

Continue full collaboration with faculty including pre/post tests, bibliographic or presentation evaluation, etc.

The Future of Assessmentat the Libraries

Hmmmmmmm……. Refine techniques that work for our institution Work to develop our instruction program

Instruction plan Collaboration Institutional requirements ???????????

Thank You for Your Attention!