Date post: | 29-Apr-2018 |
Category: |
Documents |
Upload: | nguyendien |
View: | 222 times |
Download: | 4 times |
Evaluating library assessment capabilities
A measure of effectiveness
Simon Hart
University of Otago Library
Benchmarking
“…the process of identifying best practices and learning
from others.
…enables a better understanding of practice, process or
performance and provides insights into how improvements
might be made.”
Jackson, N. (2001) Benchmarking in UK HE: an overview
Quality Assurance in Education, 9 (4), 218 – 235.
Two key problems in academic
library assessment:
the failure of libraries to select measures that
demonstrate a contribution to the strategic plan of the
institution
the selection of performance measures that are not
meaningful to campus administration.
Matthews, J. R. (2007). Library assessment in higher education.
Westport CT: Libraries Unlimited
How do we measure our effectiveness?
How do we assess our assessment?
How could we frame our assessment to
identify an improvement path?
What should a mature approach to
library assessment look like?
Detail the processes used to gather data that
informed the development of the project.
Detail the performance measures used to
evaluate the success of the project.
Outline plans you have for the ongoing evaluation
of the effectiveness of the projects outcomes.
Initial
Repeatable
Defined
Quantitatively
managed
Continuous
improvement
5
4
3
2
1
optimized
measured
confirmed
documented
ad hoc
Capacity Maturity Model
Paulk, M. C., Curtis, B., Chrissis, M. B. and Weber, C. V. (1993). Capability maturity model,
version 1.1, Institute for Software Research Paper 7.
Becker, J., Knackstedt, R., & Pöppelbuß, D. W. I. J. (2009). Developing
maturity models for IT management. Business & Information Systems
Engineering, 1(3), 213-222.
Procedure model for developing maturity models
Need identified
Process built in
Comparison made
Design & test model
Frankie Wilson : The Quality Maturity Model
Wilson, F. (2013). The quality maturity model: assessing organisational quality
culture in academic libraries. Doctoral dissertation, Brunel University, School of
Information Systems, Computing and Mathematics.
Bakkalbasi, N., Sundre, D. L., & Fulcher, K. F. (2012). Assessing assessment: A
framework to evaluate assessment practices and progress for library collections and
services. Proceedings of the 2012 Library Assessment Conference. (pp. 533-537).
Stages of Assessment cycle
Assessment Capability Maturity Model
Stages of the Assessment Cycle
Leve
ls o
f ca
pab
ility
Methods & data collections: Gathering feedback
Continuous Assessment of methods and data reuse is used to improve feedback gathering exercises
Managed Data from defined feedback methods is structured for use and reuse.
DefinedA range of repeatable methods are used to gather feedback
Repeatable Feedback is proactively sought from sub-set of clients for specific purposes
Ad hoc Feedback is client initiated and not collated
Amos, H. & Hart, S. (2013). International collaboration for quality: A case
study of fostering collaboration across an international network of
University Libraries through an activity based benchmarking project,
Proceedings of the 34th Annual IATUL conference.
Hart, S. & Amos, H. (2014). The development of performance measures
through an activity based benchmarking project across an international
network of academic libraries. Performance Measurement and Metrics,
15(1/2), 58-66.
Hart, S. & Amos, H. (in press). Building on what works: towards a library
assessment capability maturity model. Proceedings of the 11th
Northumbria International Conference on Performance Measurement in
Libraries and Information Services.
ReferencesAmos, H. & Hart, S. (2013). International collaboration for quality: A case study of fostering collaboration across an international network of University Libraries through an activity based benchmarking project, Proceedings of the 34th Annual IATUL conference. Retrieved from http://docs.lib.purdue.edu/iatul/2013/papers/48/.
Bakkalbasi, N., Sundre, D. L., & Fulcher, K. F. (2012). Assessing assessment: A framework to evaluate assessment practices and progress for library collections and services. In S. Hiller, M. Kyrillidou, A. Pappalardo, J. Self, A. Yeager (Eds.) Proceedings of the 2012 Library Assessment Conference. (pp. 533-537). Charlottesville, VA: Association of Research Libraries.
Becker, J., Knackstedt, R., & Pöppelbuß, D. W. I. J. (2009). Developing maturity models for IT management. Business & Information Systems Engineering, 1(3), 213-222.
Hart, S. & Amos, H. (2014). The development of performance measures through an activity based benchmarking project across an international network of academic libraries. Performance Measurement and Metrics, 15(1/2), 58-66.
Hevner A.R., March S. T., Park J., Ram S. (2004). Design science in information systems research. MIS Quarterly 28(1):75–105.
Jackson, N. (2001). Benchmarking in UK HE: an overview. Quality Assurance in Education, 9(4), 218-235.
Matthews, J. R. (2007). The Evaluation and measurement of library services. Westport, CT: Libraries Unlimited.
Paulk, M. C., Curtis, B., Chrissis, M. B. and Weber, C. V. (1993). Capability maturity model, version 1.1, Institute for Software Research Paper 7. Retrieved from http://repository.cmu.edu/sei/171.
Sankey , M. (2014). Benchmarks for Technology Enhanced learning: a report on the refresh project 2014 and Inter-Institutional Benchmarking Summit. Retrieved from http://www.acode.edu.au/mod/resource/view.php?id=269.
Wilson, F. & Town, J. S. (2006), Benchmarking and library quality maturity, Performance Measurement and Metrics, 7(2) 75–82.
Wilson, F. (2013). The quality maturity model: assessing organisational quality culture in academic libraries. Doctoral dissertation, Brunel University, School of Information Systems, Computing and Mathematics.
Wilson, F. (2015). The Quality Maturity Model: your roadmap to a culture of quality. Library Management, 36(3), 258-267.