University of North Texas Health Science Center
Office of Strategy & Measurement
Performance Assessment Planning for
Educational Excellence and TracDat Support
Contact Information
Office of Strategy & Measurement
817‐735‐2612
2 9/22/2010
Table of Contents
Introduction .................................................................................................................................................. 4
What is Performance Assessment? ......................................................................................................... 4
The Performance Assessment Cycle ......................................................................................................... 5
Performance Assessment Levels ............................................................................................................... 5
Principles for Assessment of Student Learning ........................................................................................ 6
UNTHSC Program of Performance Assessment ........................................................................................ 7
Academic Degree Programs .......................................................................................................................... 8
Assessment Unit ........................................................................................................................................ 8
Mission .................................................................................................................................................. 8
Vision ..................................................................................................................................................... 8
Assessment Plan ....................................................................................................................................... 8
Goals – (Student Learning Goals) .......................................................................................................... 8
Performance Measures ......................................................................................................................... 9
Tactical Initiatives ............................................................................................................................... 11
Related Tactical Initiatives/Standards ................................................................................................ 11
Assessment Results ................................................................................................................................. 11
Results ................................................................................................................................................. 11
Result Status ....................................................................................................................................... 12
Actions to Close the Loop ................................................................................................................... 12
Reports .................................................................................................................................................... 12
Documents .............................................................................................................................................. 13
Academic Departments and Schools .......................................................................................................... 14
Assessment Unit ...................................................................................................................................... 14
Mission ................................................................................................................................................ 14
Vision ................................................................................................................................................... 14
Assessment Plan ..................................................................................................................................... 14
Goals ................................................................................................................................................... 14
Performance Measures ....................................................................................................................... 15
Tactical Initiatives ............................................................................................................................... 16
Related Tactical Initiatives/Standards ................................................................................................ 17
3 9/22/2010
Assessment Results ................................................................................................................................. 17
Results ................................................................................................................................................. 17
Result Status ....................................................................................................................................... 17
Actions to Close the Loop ................................................................................................................... 17
Reports .................................................................................................................................................... 18
Documents .............................................................................................................................................. 18
Academic Support Units ............................................................................................................................. 19
Assessment Unit ...................................................................................................................................... 19
Mission Statement .............................................................................................................................. 19
Vision ................................................................................................................................................... 19
Assessment Plan ..................................................................................................................................... 19
Goals ................................................................................................................................................... 19
Performance Measures ....................................................................................................................... 20
Tactical Initiatives ............................................................................................................................... 21
Related Tactical Initiatives/Standards ................................................................................................ 22
Assessment Results ................................................................................................................................. 22
Result .................................................................................................................................................. 22
Result Status ....................................................................................................................................... 22
Actions to Close the Loop ................................................................................................................... 22
Reports .................................................................................................................................................... 23
Documents .............................................................................................................................................. 23
References .................................................................................................................................................. 24
Suggested Materials .................................................................................................................................... 24
List of Academic Programs, Academic Departments and Schools, and Academic Support Units .............. 25
4 9/22/2010
Introduction
What is Performance Assessment?
Performance Assessment is the systematic and ongoing process of collecting and analyzing data for implementing data‐driven decisions as related to goals and outcomes in support of the University of North Texas Health Science Center Strategic Plan and SACS institutional effectiveness requirements. UNTHSC focuses on two major categories of expected goals:
1. Academic Goals (i.e. student learning goals);
2. Academic Support Activity Goals (i.e. administration, research, clinical affairs, and community engagement goals).
According to the Southern Association of Colleges and Schools (SACS); Institutional Effectiveness (IE) responsibilities are:
3.3 Institutional Effectiveness
3.3.1 The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)
3.3.1.1 Educational programs, to include student learning outcomes
3.3.1.2 Administrative support services
3.3.1.3 Educational support services
3.3.1.4 Research within its educational mission, if appropriate
3.3.1.5 Community/public service within its educational mission, if appropriate
(http://www.sacscoc.org/pdf/2008PrinciplesofAccreditation.pdf)
5 9/22/2010
The Performance Assessment Cycle Performance Assessment is an ongoing cycle of planning, assessment, measurement, and implementation of the data‐driven decision process. UNTHSC has designated the following timetable for the Performance Assessment cycle:
• August 1 – September 30 – all assessment units enter results and reports for the previous academic year in TracDat®; all assessment units review and revise plans (mission, goals, performance measures, targets, & tactical initiatives) in TracDat®;
• October 1‐31 – all assessment units undergo peer review of performance assessment plans and results;
• November 1‐30 – all assessment units update performance assessment plans based on peer review feedback.
Performance Assessment Levels UNTHSC recognizes three levels of organizational structure for Performance Assessment/ Institutional Effectiveness and identifies these levels of organization as:
1. Institutional (UNTHSC)
2. Summary (Vice Presidents & Deans)
3. Academic Degree Programs, Academic Departments, and Academic Support Units
Responsibilities assigned to each level are fully described in later sections. As an institution, UNTHSC will provide a Performance Assessment Report for all Principles of Accreditation as needed to the SACS Regional Accreditation Office and the SACS liaison officer to support fulfillment of the SACS Institutional Effectiveness standard 3.3.1. UNTHSC will demonstrate full and complete compliance in all areas related to Institutional Effectiveness for accreditation purposes.
Schools and departments may use TracDat® to enter performance assessment plans and results for additional levels in support of professional or program accreditation requirements as needed.
6 9/22/2010
Principles for Assessment of Student Learning The American Association for Higher Education provides “9 Principles of Good Practice for Assessing Student Learning.” A copy of these assessments can be reviewed at the following link:
http://www.academicprograms.calpoly.edu/pdfs/assess/nine_principles_good_practice.pdf
9 Principles of Good Practice for Assessing Student Learning”
1. “The assessment of student learning begins with educational values.
2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
5. Assessment works best when it is not episodic.
6. Assessment fosters wider improvement when the representatives from across the educational community are involved.
7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
9. Through assessment, educators meet responsibilities to students and to the public.
(American Association for Higher Education, 1996)
7 9/22/2010
UNTHSC Program of Performance Assessment The general assumptions for the UNTHSC Program of Performance Assessment within the Office of Strategy & Measurement are:
A. The overarching goal for the program of performance assessment is to continually improve student learning, degree program effectiveness, and generate support for the UNTHSC Strategic Plan;
B. Results generated from assessment are for degree program, academic department, school, and academic support unit improvements;
C. Results are not to be used for evaluation in matters of tenure, promotion, merit review, or any other human resource decision process;
D. Assessment activities will be embedded in all academic and academic support expected goals;
E. The data‐driven decision process relies on the measurement and assessment of all expected goals to validate any and all changes in degree programs and academic support activities, and for the periodic realignment of the UNTHSC Strategic Plan.
8 9/22/2010
Academic Degree Programs
Assessment Unit
Mission
Mission statements at the degree program level may reflect the school mission. UNTHSC degree programs should establish, as much as possible, a unique mission that reflects the purpose of the program. The mission should be a concise statement outlining the purpose of the program, who it serves, in what ways, and with what result.
Sample Mission Statements:
• The mission of the Dr.P.H. program is to prepare public health professionals for senior leadership roles that have influence on the policies, programs and institutions of public health through the analysis, development and implementation of public health policies and programs in governmental, for‐profit, and not‐for‐profit organizations to improve the health of populations in the United States and abroad.
• The Department of Physician Assistant Studies provides an exemplary graduate‐level Physician Assistant education which focuses on primary care medicine and meeting the health needs of underserved populations. Educational objectives are met through a combination of lectures, small group activities, directed student learning, and clinical experiences.
Vision Vision statements at the program level may reflect either the departmental vision statement or the school vision statement. Sample Vision Statements:
• To become one of the Top 10 schools of public health in the nation
• To be a model of excellence and leadership in PA education
Assessment Plan
Goals – (Student Learning Goals)
Student learning goals “describe what the students will think (affective), know (cognitive), or do (behavioral/performance) when they have completed a degree program” (Nichols & Nichols, 2005, p.75). Student learning goals should be constructed as a simple sentence. For example:
• DrPH students will apply public health theory, knowledge, and skills in a practice setting.
In alignment with the model from Nichols et al. (2005), this sample student learning goal states that the student will be able to apply what they have learned in the classroom during their
9 9/22/2010
public health residency. Another example is one from Physician Assistant Studies in which the student demonstrates patient care skills.
• Students will demonstrate ability to work effectively with physicians and other healthcare professionals in providing patient‐centered care.
Student learning goals are simple statements about the skills bank that a student will possess once he/she completes the degree program. Each student learning goal should identify a singular outcome. UNTHSC degree programs will measure 5‐7 outcomes each academic year. A “Student Learning Outcome Generator” is available for authenticated users on the UNTHSC intranet at: http://www.hsc.unt.edu/qep/slo.html
Performance Measures
Performance Measure Type
Once the student learning goal is identified, a Performance Measure must be constructed in order to effectively measure the outcome. A drop‐down Performance Measure Type category box has been provided in TracDat® with the following options and definitions:
Performance Measure Type Description Activity Volume Summing or averaging the number of events or occurrences Benchmarking Comparing processes or results for best practices or
performance to similar activities inside or outside the assessment unit
Capstone Summary course or educational experience in which students demonstrate they have achieved learning goals
Comprehensive Exam End of course, degree, or rotation exam in which students demonstrate they have achieved learning goals
Efficiency/Effectiveness How well a process addresses its intended purpose, expressed as a ratio, percent, increase/decrease, etc.
Exit Interview Interviews with students to assess achievement of learning goals
Focus Group Group interview with students, constituents, or beneficiaries to assess achievement of goals
Government/Professional Standards
Comparing processes or results for best practices or performance to government or professional standards
Grant Application Submitted grant application in which students demonstrate achievement of learning goals
Internship/Residency/Practicum Evaluation
Evaluation of student performance for internship, residency, practicum, etc.
Licensure Exam Comprehensive exam administered by professional licensing body to assess achievement of learning goals
Master’s Project Scholarly project for master’s level students Oral/Written Examination Oral or written examination that assesses student's
10 9/22/2010
Performance Measure Type Description achievement of learning goals
OSCE Objective Structured Clinical Examination used to test clinical skill performance and competence
Performance Academic: an event, e.g., presentation, written assignment, or project, in which students demonstrate achievement of learning goals
Portfolio Evaluation of a collection of students' work to assess achievement of learning goals
Supervised Practice/Preceptor Evaluation
Evaluation of student performance for supervise practice, preceptorship, rotation, etc.
Survey Collection of quantitative or qualitative data from stakeholders to assess achievement of goals
Thesis/Dissertation Independent research project in which students demonstrate achievement of learning goals
Performance Measure and Target
Once the Performance Measure Type has been selected, the Performance Measure needs to be defined. An example from a public health degree program is provided.
• Goal Name – Application of knowledge, theory, and practice • Goal – Student will apply public health theory, knowledge, and skills in a practice setting • Performance Measure Type – Internship/preceptor evaluation • Performance Measure –Evaluation of student application of public health theory,
knowledge, and skills by public health residency supervisor • Target – 90% of students will be rated as meeting or exceeding expectations for
application of public health theory, knowledge, and skills in a practice setting
A second example is provided from an outcome in a physician assistant studies program.
• Goal Name – Patient care • Goal – Students will demonstrate ability to work effectively with physicians and other
healthcare professionals in providing patient‐centered care • Performance Measures Type – OSCE • Performance Measure – Students will demonstrate advancement and maintenance of
their clinical skills necessary to successfully pass the summative evaluation by successfully passing Objective Structured Clinical Evaluations (OSCE's) throughout the curriculum
• Target – 95% first time student pass rate on OSCE’s
It is important to attach a rubric assigned to the criterion which describes the assessment/ scoring process. Additionally, a copy of any survey instruments, such as an exit interview,
11 9/22/2010
should be attached and assigned as well. These can be uploaded under the Documents tab and then linked to the appropriate Performance Measure.
Tactical Initiatives
Tactical Initiatives are activities undertaken to achieve a goal and are based on results compared to the desired target performance level.
• Goal – DrPH students will apply public health theory, knowledge, and skills in a practice setting
• Tactical Initiative – Incorporate higher order thinking skills approaches to improve students ability to apply public health theory, knowledge, and skills
• Goal – Students will demonstrate ability to work effectively with physicians and other healthcare professionals in providing patient‐centered care
• Tactical Initiative – Incorporate higher order thinking skills approaches to improve students ability to demonstrate effective patient care skills
Related Tactical Initiatives/Standards
Tactical initiatives from the respective school or accreditation standards may be linked to the degree program goals via check box format in the Related Tactical Initiatives/Standards tab within the Assessment Plan tab. Tactical initiatives/standards are to be checked where the degree program goal demonstrates linkage. More than one tactical initiative/standard may be checked for a single goal.
Assessment Results
Results
Assessment results are report data recorded for the Goal in the active fiscal year. Data may be recorded as tables, charts, rubrics, performance (not GPA or course grades), or other means of data expression as related to the Goal. For example:
• After compilation of 16 public health residency evaluations, data show that 100% of the students either met or exceeded expectations for student application of public health theory, knowledge, and skills in a practice setting. A copy of the results has been attached in the document repository for review.
• All 28 students in the graduating Class of 2008 completed the summative evaluation. One student (3.5%) required remediation of one of six components of the evaluation, but successfully completed that remediation prior to graduation. This equates to a 96% first time pass rate on the summative evaluation.
Any documentation of results, e.g., spreadsheet, report, etc. may be uploaded under the Documents tab and then linked to the Result.
12 9/22/2010
Result Status
A drop‐down menu is available in the system for Result Status. Select the option provided:
1. Inconclusive – results not sufficient to allow determination of meeting target
2. Target Met – results indicate target was met or exceeded
3. Target Not Met – results indicate target was not met
Actions to Close the Loop
Actions are activities or changes planned in response to individual performance measure results in order to improve or maintain performance. Actions loop may form the basis for Tactical Initiatives for the next performance assessment cycle.
Follow‐Up to Actions are notes related to activities undertaken or changes made for individual performance measure actions.
• Result – After compilation of 16 public health residency evaluations, data show that 100% of the students either met or exceeded expectations for student application of public health theory, knowledge, and skills in a practice setting. A copy of the results has been attached in the document repository for review
• Action – The DrPH committee will review the public health residency rubric to ensure that residency supervisors are able to objectively evaluate students’ practice skills
• Follow‐up to Actions – The DrPH committee reviewed the public health residency rubric and met with a sample of public health residency supervisors. It was determined that the current rubric is effective in evaluating students’ practice skills
• Result – All 28 students in the graduating Class of 2008 completed the summative evaluation. One student (3.5%) required remediation of one of six components of the evaluation, but successfully completed that remediation prior to graduation. This equates to a 96% first time pass rate on the summative evaluation
• Action – The PA curriculum committee will review the need for changes to the curriculum to ensure the program continues to meet or exceed the target for this goal
• Follow‐up to Actions – The PA curriculum committee reviewed the curriculum and recommended that faculty increase their use of instructional strategies and techniques that are designed to increase student higher order thinking
Reports The “Assessment Plan” and “Unit Assessment – Four Column Report” are the preferred reports. The Assessment Plan shows the information entered for the assessment plan, e.g., mission, vision, goals, tactical initiatives, performance measures, and targets. The Unit Assessment – Four Column Report includes the information from the Assessment Plan along with the results and any action plans in a four column report. For academic end of year reports, please print only “active” Goals in the report process.
13 9/22/2010
Documents Documents may be uploaded under the document tab. Once a document is uploaded, it can be linked to support a Performance Measure or an Assessment Result.
14 9/22/2010
Academic Departments and Schools
Assessment Unit
Mission
Mission statements of academic departments and schools may reflect the institutional mission. UNTHSC departments and schools should establish, as much as possible, a unique mission that reflects the purpose of the department or school. The mission should be a concise statement outlining the purpose of the department, who it serves, in what ways, and with what result.
Sample Mission Statements:
• The Mission of the Department of Epidemiology is to provide an outstanding comprehensive education and mentoring in Epidemiology for students, to prepare students for a career in epidemiologic practice, research and academia, to conduct high quality research that will contribute to improving the health of populations, and to provide service to the local, national and international communities.
• The mission of the Graduate School of Biomedical Sciences is to advance medical sciences through innovative research and develop outstanding biomedical and clinical research scientists.
Vision Vision statements at the department or school level may reflect either the either the departmental vision statement or the school vision statement. Sample Vision Statements:
• To become one of the premier departments of Epidemiology in the nation
• To become a top 10 biomedical sciences school
Assessment Plan
Goals
Goals at the department/school level should be “leading the faculty in conducting assessment activities resulting in substantive program improvement” (Nichols & Nichols, 2005, p.58). This may be accomplished by the data‐driven decision process which entails:
• What goals are to be accomplished by the department/school? • How are the goals to be accomplished by the department/school? • When is the goal to be completed by the department/school? • What resources will be needed to enable the department/school to accomplish these
goals?
15 9/22/2010
An example of a Goal from a department in the School of Public Health might be:
• The Epidemiology Department will increase student enrollment by 25% over the next three years.
The goal is to increase student enrollment within the department with an expressed timeline of three years. Another example would be:
• The Graduate School will produce quality research that results in publications, extramural funding and national standing.
The goal is to gain national standing in research by producing quality research publications and obtaining extramural funding.
Performance Measures
Performance Measure Type
Once the goal is identified, a Performance Measure must be constructed in order to effectively measure the outcome. A drop‐down Performance Measure Type category box has been provided in TracDat® with the following options and definitions:
Performance Measure Type Description Activity Volume Summing or averaging the number of events or
occurrences Benchmarking Comparing processes or results for best practices or
performance to similar activities inside or outside the assessment unit
Efficiency/Effectiveness How well a process addresses its intended purpose, expressed as a ratio, percent, increase/decrease, etc.
Focus Group Group interview with students, constituents, or beneficiaries to assess achievement of goals
Government/Professional Standards Comparing processes or results for best practices or performance to government or professional standards
Project/Activity Completion Completion, implementation, or achievement of a project, activity, or process
Survey Collection of quantitative or qualitative data from stakeholders to assess achievement of goals
Performance Measure and Target
Once the Performance Measure Type category has been selected, the Performance Measure needs to be defined. An example from an academic department is provided.
16 9/22/2010
• Goal Name – Student Enrollment • Goal – The Epidemiology Department will increase student enrollment by 25% over the
next three years • Performance Measure Type – Activity Volume • Performance Measure – Number of enrolled students (headcount and FTE) based on
data obtained from the Office of the Registrar • Target – 25% increase in student enrollment with an annual increase of approximately
8%
A second example is provided from a school research goal.
• Goal Name – Extramural Research Funding • Goal Expected Outcome – The Graduate School will produce quality research that
results in publications, extramural funding and national standing • Performance Measure Type – Activity Volume • Performance Measure – Total dollar value of grant applications submitted and
extramural research funding awarded as reported by Office of Grant and Contract Management
• Target ‐ $145.7 million in grant applications submitted and $26.2 million in research funding awarded
It is important to attach any supporting information for the criterion which describes the assessment/ scoring process. Additionally, a copy of any survey instruments, such as a satisfaction survey, should be attached and assigned as well. These can be uploaded under the Documents tab and then linked to the appropriate Performance Measure.
Tactical Initiatives
Tactical Initiatives are activities undertaken to achieve a goal and are based on results compared to the desired target performance level.
• Goal – The Epidemiology Department will increase student enrollment by 25% over the next three years.
• Tactical Initiative – Identify best sources of quality epidemiology graduate students and target recruitment efforts.
• Goal – The Graduate School will produce quality research that results in publications, extramural funding and national standing.
• Tactical Initiative – Provide research development efforts through mentorship and formal training experiences.
17 9/22/2010
Related Tactical Initiatives/Standards
Tactical initiatives from the respective school or accreditation standards may be linked to the department program goals via check box format in the Related Tactical Initiatives/Standards tab within the Assessment Plan tab. Schools will link to UNTHSC Tactical Initiatives. Tactical initiatives/standards are to be checked where the degree program goal demonstrates linkage. More than one tactical initiative/standard may be checked for a single goal.
Assessment Results
Results
Assessment results are report data recorded for the Goal in the active fiscal year. Data may be recorded as tables, charts, rubrics, performance (not GPA or course grades), or other means of data expression as related to the Goal. For example:
• For the current year, Epidemiology enrollment increased 10%, which exceeds the annual target to meet or exceed the 25% increase over three years.
• The Graduate School submitted $152 million in grant applications and received $28.5 million in grant funding, which exceeded current year targets
Any documentation of results, e.g., spreadsheet, report, etc. may be uploaded under the Documents tab and then linked to the Result.
Result Status
A drop‐down menu is available in the system for Result Status. Select the option provided:
1. Inconclusive – results not sufficient to allow determination of meeting target
2. Target Met – results indicate target was met or exceeded
3. Target Not Met – results indicate target was not met
Actions to Close the Loop
Actions are activities or changes planned in response to individual performance measure results in order to improve or maintain performance. Actions loop may form the basis for Tactical Initiatives for the next performance assessment cycle.
Follow‐Up to Actions are notes related to activities undertaken or changes made for individual performance measure actions.
• Result – For the current year, Epidemiology enrollment increased 10%, which exceeds the annual target to meet or exceed the 25% increase over three years
• Action – Epidemiology faculty and OSAS will conduct a review of feeder schools and programs to determine the best opportunities for recruitment efforts
18 9/22/2010
• Follow‐up to Actions – Epidemiology faculty and OSAS identified feeder schools and programs to target and developed recruitment plans
• Result – The Graduate School submitted $152 million in grant applications and received $28.5 million in grant funding, which exceeded current year targets
• Action – The Graduate School will conduct a needs assessment of faculty members to determine faculty development needs
• Follow‐up to Actions – The Graduate School conducted a faculty development needs assessment and used the results to establish faculty development priorities for the upcoming year
Reports The “Assessment Plan” and “Unit Assessment – Four Column Report” are the preferred reports. The Assessment Plan shows the information entered for the assessment plan, e.g., mission, vision, goals, tactical initiatives, performance measures, and targets. The Unit Assessment – Four Column Report includes the information from the Assessment Plan along with the results and any action plans in a four column report. For academic end of year reports, please print only “active” Goals in the report process.
Documents Documents may be uploaded under the document tab. Once a document is uploaded, it can be linked to support a Performance Measure or an Assessment Result.
19 9/22/2010
Academic Support Units
Assessment Unit
Mission Statement
Mission statements of administrative support units may reflect the institutional mission statement. UNTHSC administrative support departments and divisions should establish, as much as possible, a unique mission statement that reflects the purpose of the department or division. The mission statement should be a concise statement outlining the purpose of the department, who it serves, in what ways, and with what result.
Sample Mission Statements:
• Facilities Management's mission is to create and provide a UNTHSC campus with aesthetically pleasing, clean, well maintained, energy efficient and appropriate facilities and grounds to support the institutional mission; while providing responsive, friendly, and efficient customer service.
• The mission of the Development Office is to increase support for the UNTHSC through individual major gifts, foundation grants, annual events, and alumni and friends annual giving. To strive to encourage support for priority areas while maintaining a donor‐centric development philosophy.
Vision Vision statements at the department or school level may reflect either the either the departmental vision statement or the school vision statement. Sample Vision Statements:
• To provide customer focused, high quality facilities and services that supports UNTHSC becoming a top 10 health science center
• To build community and financial support that enables UNTHSC to become a top 10 health science center
Assessment Plan
Goals
Goals at the department/school level should be “leading the faculty in conducting assessment activities resulting in substantive program improvement” (Nichols & Nichols, 2005, p.58). This may be accomplished by the data‐driven decision process which entails:
• What goals are to be accomplished by the department? • How are the goals to be accomplished by the department? • When is the goal to be completed by the department?
20 9/22/2010
• What resources will be needed to enable the department to accomplish these goals?
An example of a Goal from Facilities Management might be:
• Improve performance in Facilities Management through ongoing process improvement and quality enhancement activities.
The goal is to use ongoing process improvement and quality enhancement activities to improve performance. Another example in an academic support unit would be:
• Engage alumni in the life of their school and increase alumni philanthropy to their school and UNTHSC.
The goal is to increase alumni involvement and philanthropy for the schools they attended.
Performance Measures
Performance Measure Type
Once the goal is identified, a Performance Measure must be constructed in order to effectively measure the outcome. A drop‐down Performance Measure Type category box has been provided in TracDat® with the following options and definitions:
Performance Measure Type Description Activity Volume Summing or averaging the number of events or
occurrences Benchmarking Comparing processes or results for best practices or
performance to similar activities inside or outside the assessment unit
Efficiency/Effectiveness How well a process addresses its intended purpose, expressed as a ratio, percent, increase/decrease, etc.
Focus Group Group interview with students, constituents, or beneficiaries to assess achievement of goals
Government/Professional Standards Comparing processes or results for best practices or performance to government or professional standards
Project/Activity Completion Completion, implementation, or achievement of a project, activity, or process
Survey Collection of quantitative or qualitative data from stakeholders to assess achievement of goals
Performance Measure and Target
Once the Performance Measure Type category has been selected, the Performance Measure needs to be defined. An example from an academic support department is provided.
21 9/22/2010
• Goal Name – Performance Improvement • Goal – Improve performance in Facilities Management through ongoing process
improvement and quality enhancement activities • Performance Measure Type – Survey • Performance Measure – Percent of customer who are satisfied or highly satisfied with
the overall performance of Facilities Management as measured by annual customer satisfaction survey
• Target – 75% of customers are satisfied or highly satisfied with overall performance of Facilities Management
A second example is provided from an alumni engagement goal.
• Goal Name – Alumni Engagement • Goal Expected Outcome – Engage alumni in the life of their school and increase alumni
philanthropy to their school and UNTHSC • Performance Measure Type – Activity Volume • Performance Measure – The number of alumni who contribute to the annual campaign
out of the total number of contactable alumni • Target – 800 donors/3800 contactable alumni = 21% participation
It is important to attach any supporting information for the criterion which describes the assessment/ scoring process. Additionally, a copy of any survey instruments, such as a satisfaction survey, should be attached and assigned as well. These can be uploaded under the Documents tab and then linked to the appropriate Performance Measure.
Tactical Initiatives
Tactical Initiatives are activities undertaken to achieve a goal and are based on results compared to the desired target performance level.
• Goal – Improve performance in Facilities Management through ongoing process improvement and quality enhancement activities
• Tactical Initiative – Adopt the Plan Do Check Act as the standard Facilities Management process improvement model and provide process improvement training for Facilities Management personnel
• Goal – Engage alumni in the life of their school and increase alumni philanthropy to their school and UNTHSC
• Tactical Initiative – Create meaningful events and opportunities designed to engage alumni in the life of their school
22 9/22/2010
Related Tactical Initiatives/Standards
Tactical initiatives from the respective summary unit may be linked to the department via check box format in the Related Tactical Initiatives/Standards tab within the Assessment Plan tab. Tactical initiatives are to be checked where the department goal demonstrates linkage. More than one tactical initiative may be checked for a single goal.
Assessment Results
Result
Assessment results are report data recorded for the Goal in the active fiscal year. Data may be recorded as tables, charts, rubrics, performance (not GPA or course grades), or other means of data expression as related to the Goal. For example:
• 80% of the customer satisfaction survey respondents were satisfied or highly satisfied with the overall performance of the Facilities Management Department
• 810 of 3800 (21.3%) contactable alumni contributed to the annual campaign
Any documentation of results, e.g., spreadsheet, report, etc. may be uploaded under the Documents tab and then linked to the Result.
Result Status
A drop‐down menu is available in the system for Result Status. Select the option provided:
1. Inconclusive – results not sufficient to allow determination of meeting target
2. Target Met – results indicate target was met or exceeded
3. Target Not Met – results indicate target was not met
Actions to Close the Loop
Actions are activities or changes planned in response to individual performance measure results in order to improve or maintain performance. Actions loop may form the basis for Tactical Initiatives for the next performance assessment cycle.
Follow‐Up to Actions are notes related to activities undertaken or changes made for individual performance measure actions.
• Result – 80% of the customer satisfaction survey respondents were satisfied or highly satisfied with the overall performance of the Facilities Management Department
• Action – Facilities Management staff will review customer survey results to determine opportunities for improvement
• Follow‐up to Actions – Facilities Management staff analyzed responses to open‐ended survey questions and identified three opportunities for service improvement
23 9/22/2010
• Result – 810 of 3800 (21.3%) contactable alumni contributed to the annual campaign • Action – Development will work with representatives from each school to investigate
possible incentives to increase alumni engagement • Follow‐up to Actions – Development is working to develop annual activities for each
school that will be designed to increase the number of alumni who participate in alumni activities and contribute to the annual campaign
Reports The “Assessment Plan” and “Unit Assessment – Four Column Report” are the preferred reports. The Assessment Plan shows the information entered for the assessment plan, e.g., mission, vision, goals, tactical initiatives, performance measures, and targets. The Unit Assessment – Four Column Report includes the information from the Assessment Plan along with the results and any action plans in a four column report. For academic end of year reports, please print only “active” Goals in the report process.
Documents Documents may be uploaded under the document tab. Once a document is uploaded, it can be linked to support a Performance Measure or an Assessment Result.
24 9/22/2010
References American Association for Higher Education (1996). AAHE assessment forum: 9 Principles of good practice for assessing student learning. Retrieved June 03, 2010 from http://www.academicprograms.calpoly.edu/pdfs/assess/nine_principles_good_practice.pdf
Nichols, J.O., & Nichols, K.W. (2005). A road map for improvement of student learning and support services through assessment. New Jersey: Agathon Press.
Suggested Materials Ammons, D. N. (Ed.) (2008). Leading performance management. Washington, D.C.: ICMA Press.
Berk, R.A., (2006). Thirteen strategies to measure college teaching. Virginia: Stylus Publishing, LLC.
Bers, R.H., & Seybert, J.A. (1999). Effective Reporting. Florida: Association for Institutional Research.
Gronlund, N.E., & Brookhart, S.M. (2009). Gronlund’s writing instructional objectives (8th ed.). New Jersey: Pearson Education.
Hatry, H. (2006). Performance measurement: Getting results. Washington, D.C.: The Urban Institute Press.
Kuh, G.D., Kinzie, J., Schuh, J.H., & Whitt, E.J. (2005). Student success in college: Creating conditions that matter. California: Jossey‐Bass.
Nichols, J.O., & Nichols, K.W. (2005). A road map for improvement of student learning and support services through assessment. New Jersey: Agathon Press.
Suskie,L. ((2004). Assessing student learning: A common sense guide. Massachusetts: Anker Publishing Company.
Walvoord, B.E. (2004). Assessment clear and simple: A practical guide for institutions, departments, and general education. California: Jossey‐Bass.
25 9/22/2010
List of Academic Programs, Academic Departments and Schools, and Academic Support Units Academic Programs Graduate School of Biomedical Sciences MS, Biomedical Sciences, Specialized MS, Biomedical Sciences, Traditional PhD, Biomedical Sciences School of Health Professions MPAS, Physician Assistant Studies Doctor of Physical Therapy Program School of Public Health Master of Health Administration Master of Public Health MPH, Biostatistics MPH, Envl & Occupational Health MPH, Epidemiology MPH, Health Management & Policy MPH, Community Health DrPH, Public Health PhD, Public Health Sciences Texas College of Osteopathic Medicine DO Program
Academic Departments and Schools Graduate School of Biomedical Sciences Cell Biology & Anatomy Forensic & Investigative Genetics Integrative Physiology Molecular Biology and Immunology Outreach, GSBS Pharmacology and Neuroscience School of Health Professions Physical Therapy Department Physician Assistant Studies Department School of Public Health Biostatistics Environmental & Occupational Health Epidemiology Health Management & Policy Social & Behavioral Sciences Texas College of Osteopathic Medicine Community Medicine Family Medicine Internal Medicine OB/GYN Orthopedics and Podiatry Osteopathic Manipulative Medicine Pediatrics Psychiatry and Behavioral Health Surgery
26 9/22/2010
Academic Support Units Administration Human Capital Management Faculty Affairs Administration Student Affairs Enrollment Services Student Services Academic and Institutional Resources & Technology Biomedical Communications Center for Learning and Development Gibson D Lewis Library Information Technology Services PACE Records Information Management Clinical Affairs and Business Development UNT Health Community Engagement Community Relations Development Marketing & Communications Operations Central Services Facilities Management HSC Police Safety Office Finance Accounting and Financial Reporting Budget & Financial Planning Fiscal Management Payroll Purchasing and Payment Services Office of Research Laboratory Animal Medicine Office for the Protection of Human Subjects Office of Clinical Trials Office of Grant & Contract Management Research Compliance Strategic Research Initiatives Technology Transfer & Commercialization