Date post: | 23-Dec-2015 |
Category: |
Documents |
Upload: | brenda-blankenship |
View: | 219 times |
Download: | 0 times |
Two birds with one evaluation
How to assess your services and publish too?
Suzana Sukovic, Jane Van Balen,Ashley England & Janet Chelliah
Research for LIS Practitioners, February 2011
Overview
• Part 1. Evaluation (with a touch of mystery)
• Part 2. Information literacy evaluation: Bachelor of Nursing Self Discovery Learning 2010-
• Part 3. Discussion
Part 1: Evaluation with a touch of mystery
Presented by
Suzana Sukovic
Evaluation
Assessment Librarian
The University of Tennessee Libraries
6 January 2011
The University of Tennessee, Knoxville (UTK) Libraries invites applications for the newly created position of Assessment Librarian. The Assessment Librarian fosters a culture of assessment within the University Libraries and leads the design and implementation of a successful, sustainable assessment program. Engaging staff throughout the Libraries, the Assessment Librarian analyzes systems and services for continuous improvement.
Evaluation• Critical part of evidence based practice
and action research
Cycles of action research (State of NSW, Department of Education and Training, Professional Learning and Leadership Development Directorate, 2010, Action research in education: guidelines, 2nd ed, p. 4)
Evaluation: approach
Where to start? • Strategic direction• Particular purpose (e.g., solving a
problem, grant application, part of a larger project)
• Opportunity (e.g., additional resource, a particularly suitable/innovative project)
Evaluation: approach
PurposeWhy evaluate?
Evaluation focusWhich aspect to evaluate?
EvidenceWhat is the evidence?
Data-gathering methods How to obtain evidence?
Analysis & interpretationHow to interpret evidence?
•Service improvement•Demonstrating value etc.
•Development of students’ IL skills•Cost benefit•Improvement of clients’ satisfactionetc.
Clients’ ability to•describe A•do B•apply A&B in new situationsetc.
•Tests•Experiments•Observation•Interviews with teachers & studentsetc.
Consider•Criteria•Benchmarks•Interpretation framework & methods•Software
Example: Evaluation of an information literacy program
Evaluation: approach
• Brainstorm• Focus on– the best fit for the purpose–most effective & efficient approach– ‘maximizing potential’
• ‘Cutting salami’ approach
Maximising evaluation projects
Make a snow ball effect!
http://blog.balancedspirituality.com/wp-content/uploads/2009/12/monkey-snowball.jpg
Maximising evaluation projects
• Choose a general theme – relates to strategic direction, a large or long-lasting project, a significant problem or opportunity
• Keep research on mind • LINK
– several small projects– with projects in other departments/organisations– with your skills, knowledge and experiences– with literature
There is nothing so practical as a good theory (Kurt Lewin, 1951)
• Publish & present – to your department, at conferences, in professional and academic journals
Wire Guys Chillin at the Baltic, New Castle UK / Marc Hartley
http://vi.sualize.us/view/invertebra/86f69425bccb521a251cb3eb8053c53d/
Part 2: Information literacy evaluation Bachelor of Nursing Self Discovery Learning 2010-
Presented by
Jane Van Balen
Background
• Directive/Need for library to better evaluate Information Literacy programs generally
• Newly revised Bachelor of Nursing course - a worthy pilot because:– New course – fresh opportunity– Follow-up possible over next years of course– InfoLit training embedded by faculty– Self Discovery approach suggested by
faculty
Background
• New course has a whole day allocated for Library information literacy as part of the subject “Workshops in Practice Readiness”.
• The Faculty suggested a self discovery approach for one of the components of the training which the library was already developing ideas for.
• Several factors came together fortuitously which allowed us to maximise this opportunity.
• We were allowed freedom to incorporate elements to better enable us to use for research. Eg Pre-Test addition.
Project objectives/directives
• To determine whether Information Literacy programs offered by the library are effective in enabling students to become independently information literate.
• To determine whether they continue developing over time after having received information literacy training.
Library Program – First year BNurs
• Online Pre-test : students could do this before they arrived or in a classroom at the start of the day. Not assessed by their tutor, but formed some of our data.
• Self discovery questionnaire: students grouped in pairs could answer a range of questions about library services. Token prizes were offered to the first pair who finished, and informal feedback was given to many as they returned their completed answers.
• Databases session: presented by their tutors using library’s lesson plan and handouts. Tutors attended a “Train the trainer” session with library staff.
• Online Post-test: This was compiled by the subject co-ordinator and was assessed. Library had access to overall results.
Data collected to date
• Results from the pre-test. Available online.
• Answers from the Self Discovery questionnaire. Written form only.
• Results from the post-test. Available online
What does the data tell us?
• If the overall scores from the post test were higher than the pre test, can we conclude that learning took place?
• Until we follow up with the next phase of training, it will be difficult to determine our original objective.
Issues – how to deal with
• Inconsistencies:– Various quality of db session presenters– Timetabling– Not everyone did the pre-test first– Self discovery questionnaire feedback
varied considerably.
• Compiling data and interpretation
Next steps
• We will meet the same groups of students when they do an Evidence Based Practise subject in 2011
• Associated with more in-depth Databases skills session. (PICO)
• Another opportunity to assess information searching skills and determine lasting benefits of training.
Next steps
• We have a number of elements we can use to proceed:– Another online pre-test (stimulus for learning)– online tutorials on Evidence Based Practice
approach to finding specific types of articles. – Library workshop with specialist subject
Librarians– Another online post-test AND/OR obtain
access to the results of their related assessment task.
Gift from Two birds...
Diigo group Research for practitioners
• Explore• Add• Comment
Do we have your email?