TBR Academic Affairs 1
THE ACADEMIC AUDITA PROCESS OF CONTINUOUS
QUALITY IMPROVEMENT
TBR OFFICE OF ACADEMIC AFFAIRSFALL 2011
Quality is not an act, it is a habit.
Aristotle
Greek Philosopher, Scientist
and Physician (384 BC-322 BC)
TBR Academic Affairs 2
THE GUIDING PRINCIPLE
Initiated in the 1980s in United Kingdom to focus on quality of teaching in research universities
Led by Dr. William Massey of Stanford University and the Jackson Hole Higher Education Group
Applied globally in New Zealand, Sweden, & Hong Kong
Adapted by the University of Missouri system in 2002
Piloted in the Tennessee Board of Regents system in 2004 by Dr. Paula Myrick Short, Vice Chancellor for Academic Affairs, and 14 of 19 institutions
Used system-wide for performance funding and in support of institutional effectiveness in the TBR 2005 – present
TBR Academic Affairs 3
ACADEMIC AUDIT BACKGROUND A BRIEF HISTORY
4
ACADEMIC AUDIT IN THE TBR SYSTEM260 PROGRAMS AUDITED FROM 2004 - 2011
47 PROGRAMS SCHEDULED FOR 2011-12
2008 Leveraging Award: Honorable
Mention to the TBR Academic Audit
Initiative bestowed by the National
Consortium for Continuous
Improvement in Higher Education
TBR Academic Affairs 5
ACADEMIC AUDIT RECOGNITION
3.3 Institutional Effectiveness
3.3.1 The institution identifies expected outcomes,
assesses the extent to which it achieves these outcomes,
and provides evidence of improvement based on analysis
of the results in each of the following areas:
3.3.1.1 educational programs, to include student learning
outcomes
TBR Academic Affairs 66
THE ACADEMIC AUDITTHE ACADEMIC AUDIT AND INSTITUTIONAL EFFECTIVENESSAND INSTITUTIONAL EFFECTIVENESS
Accepted by THEC since 2005 as a means of program
evaluation for non-accreditable degree and certificate
programs
PF Summary Sheet provides the program’s score, which is
computed into the PF formula
New for the 2010-2015 cycle: Follow-up to previous Academic
Audit criteria
TBR Academic Affairs 77
THE ACADEMIC AUDITTHE ACADEMIC AUDIT AND PERFORMANCE FUNDINGAND PERFORMANCE FUNDING
A faculty-driven model of ongoing self-
reflection, peer feedback, collaboration, and
teamwork based on structured conversation
to improve educational quality processes in
teaching and learning … and hence student
success.
William Massy
TBR Academic Affairs 8
WHAT IS THE ACADEMIC AUDIT?
1.Define quality in terms of OUTCOMES
2.Focus on PROCESS
3.Work COLLABORATIVELY
4.Base decisions on EVIDENCE
5.Strive for COHERENCE
6.Learn from BEST PRACTICE
7.Make CONTINUOUS IMPROVEMENT a priority
TBR Academic Affairs 9
ACADEMIC AUDITUNDERLYING QUALITY PRINCIPLES
1. The Self Study – fall semester
2. The Self Study Report – due January 31, 2012
3. The Auditor Site Visit – March 19 – April 20
4. Implementation of Initiatives – ongoing
TBR Academic Affairs 10
ACADEMIC AUDIT COMPONENTS
Form Self Study Team & identify Leader
Assign key roles - focal area responsibility
Set schedule with due dates
Identify Stakeholders
Select sources of evidence
Nominate peers for Auditor Team
TBR Academic Affairs 11
CONDUCTING THE SELF STUDY
Hold conversations among faculty, stakeholders
and students around focal areas
Draft self study report
Prioritize improvement initiatives
Finalize self study report
Complete Appendices [one may be links]
Send to TBR: by Monday, January 31, 2011
TBR Academic Affairs 12
CONDUCTING THE SELF STUDY
1. Learning Objectives
2. Curriculum and Co- Curriculum
3. Teaching and Learning
4. Student Learning Assessment
5. Quality Assurance
TBR Academic Affairs 13
THE SELF STUDY PROCESS ORGANIZATION BY FOCAL AREA
1. What do we want our students to know AND
why?
2. How do we determine what our students should
understand and demonstrate (student learning
outcomes) for courses and our program of
study as a whole?
3. What processes are in place by which we
regularly evaluate our Learning Objectives?
TBR Academic Affairs 14
GUIDING QUESTIONS FOR FOCAL AREA 1:
LEARNING OBJECTIVES
1. What processes do we follow to assure
that our curriculum is coherent (sensibly
sequenced)?
2. What activities beyond the classroom do
we promote to reinforce our curriculum?
TBR Academic Affairs 15
GUIDING QUESTIONS FOR FOCAL AREA 2: CURRICULUM AND CO-CURRICULUM
1. What do we do collaboratively to improve our
teaching?
2. How do we connect with our students to be
assured that learning is taking place?
3. How do we identify and share best practices in
the teaching of our subject?
TBR Academic Affairs 16
GUIDING QUESTIONS FOR FOCAL AREA 3: TEACHING & LEARNING
1. Do our assessments of student learning
measure student mastery of our learning
objectives? [course AND program]
2. How do we use assessment data to
improve teaching and learning?
TBR Academic Affairs 17
GUIDING QUESTIONS FOR FOCAL AREA 4: STUDENT LEARNING ASSESSMENT
1. What college-wide and program-wide
practices promote quality in teaching
and learning?
2. What college-wide and program-wide
services and resources promote quality
in teaching and learning?
TBR Academic Affairs 18
GUIDING QUESTIONS FOR FOCAL AREA 5: QUALITY ASSURANCE
1.Introduction2.Overall Performance3.Performance by Focal Area4.Potential Initiatives5.Matrix of Improvement Initiatives6.(for 2nd time programs) – Follow-up
report7.Appendix
TBR Academic Affairs 19
THE SELF STUDY REPORT
1.Brief History
2.Role & Scope –
purpose/mission
3.Demographics
4.Distinguishing characteristics
5.How self-study was performedTBR Academic Affairs 20
SELF STUDY REPORT: INTRODUCTION ORGANIZATIONAL PROFILE
1.Executive summary
2.Strengths
3.Key Findings
TBR Academic Affairs 21
SELF STUDY REPORT: OVERALL PERFORMANCE
1. One to three pages on each Focal Area
2. Keep impersonal [“anonymous”]
3. Cite strengths in that area especially the
processes in place that help assure quality
4. Include one or two illustrative examples or
anecdotes
5. Identify potential areas for improvement
TBR Academic Affairs 22
SELF STUDY REPORT: PERFORMANCE BY FOCAL AREA
1.Respond to all opportunities for
improvement identified in report
2.State goals to demonstrate how
improvement in this area would
positively enhance student learning
TBR Academic Affairs 23
SELF STUDY REPORT: POTENTIAL INITIATIVES
1. Who has responsibility for implementation?
2. Who will participate in the initiative?
3. When will work begin on the initiative?
4. How long will implementation take?
5. Who and how will you chronicle results?
6. How will results be used?
TBR Academic Affairs 24
SELF STUDY REPORT: MATRIX OF IMPROVEMENT INITIATIVES
1. Select the most relevant and helpful additional
information
2. Limit hard copy Appendix to ten (10) pages
[more for graduate programs]
3. Check accuracy of each web page link provided
TBR Academic Affairs 25
SELF STUDY REPORT: APPENDIX
1. Two to four member team of peers from TBR institutions or other institutions
2. The team reviews the Self Study Report & Appendices then forms guiding questions
3. One day on-site visit including conversations with Faculty and Students
4. Immediate Feedback – oral report by the team on day of site visit (exit session)
5. Written report submitted by team
TBR Academic Affairs 26
THE AUDITOR TEAM REVIEW PROCESS
For in-depth discussion of the Academic Audit concept, process and TBR pilot:
TBR Academic Affairs 27
ACADEMIC AUDIT RESOURCES
Academic Quality Work: A Handbook for Improvement (JB - Anker Series,
2007) by William F. Massy, Steven W. Graham, & Paula Myrick Short
ISBN 978-1-933371-23-8
For additional information contact:
Dr. Randy Schulte
Assistant Vice Chancellor for Academic Affairs615-365-1505 FAX 615-366-3903