The Board’s Role in Overseeing Educational Quality
Peter Eckel, AGBVicki Golich, Metropolitan State University
DenverJeremy Haefner, Rochester Institute of
Technology
Overseeing Educational Quality
The Role of the Board
The Board? (Isn’t ed quality a faculty responsibility?)
• Fiduciary responsibility (parallel to financial audit)
• Responsibility ensure that decision makers have best tools and data available
• Ultimate responsibility for soundness and integrity of the institution’s programs
The Curriculum is the Faculty’s Responsibility…
1. Ensure that the institution has an appropriate set of learning outcomes statements
2. Ensure that efforts to determine the effectiveness of teaching and learning are in place and ongoing
3. Ensure that institutions use the data they collect for improvement
The Board’s Role is to Remind Them of That Responsibility
Peter Ewell. Making the Grade. AGB
Boards and Student LearningIncreasingly concerned;
but continually perplexed
6
What Boards Hear: The Public Story
What Boards See: How Students (and Families) Feel
Discussion: Your Board and Student Learning Outcomes• Does it spend too much; too little; or
just enough time on student learning outcomes?
• What is the relative balance of time discussing learning vs money matters?
• Does the board spend more time on student learning now vs. 5 years ago?
Most Boards Don’t Do Enough
AGB-Teagle Project: • Drake University (IA)• Metropolitan State University of Denver• Morgan State University (MD)• Salem State University (MA)• St. Olaf College (MN)• Rhodes College (TN)• Rochester Institute of Technology (NY)• Valparaiso University (IN)
A quiz: Does your board……
• Know the institution’s learning outcomes goals? – (Know that you have goals???)
• Have a dashboard learning outcomes (direct/indirect outcomes) ?
• Discuss ed quality with faculty?• See and discuss academic program reviews?• Set aside time for educational quality?
Don’t Be Surprised:Common Board Difficulties • Fish (accountants?) out of water
• Impatience/Lack of time: Academic issues take time
• Unfamiliar language of academic assessment (I thought NSSE was a sea monster…. And NIOLA was a town in Hawaii…..)
• Evidence of academic quality is often ambiguous and hard to interpret and use
Two Approaches
•Rochester Institute of Technology
•Metropolitan State University of Denver
14
Academic quality and the Board of Trustees from the
RIT point of view
April 2014
15
AGENDA
Context for a trilogy discussion on academic quality
The Academic Quality Dashboard
The Teagle AGB project
Education Core Committee By-laws
Plan of Work for AY 2013-2014
CONTEXT
The context for the conversation• 2008 – Provost hired with
mandate to engage the board
• Historically BOT very engaged with financial fiduciary role
• 2009 – Great recession begins to manifest
• Concern for academic quality in tough financial time
Partnership with Education committee• Charlie Brown, Chair,
Education Core Committee
• Devised a ‘trilogy’ of conversations on academic quality with the Education Core Committee– 2011: April, November, July
19
Objectives
To ensure academic quality is central to our roles and responsibilities• Be well informed• Understand fiduciary
responsibility is linked to academic quality
To advance academic quality @ RIT• Ensure strategy and
policies are appropriate• Ensure processes are
appropriate and in place• Advocate
To provide input to Middle States• Periodic Review Report
due June 2012• Next campus visit
20
Process
April
• Setting the stage• Understanding
quality through indicators
• The reading assignment
July
• Data and processes
• Discussion• Initial take-aways
and ideas
November
• Principles, processes, and practices
• An Education Committee dashboard
21
Important tool for discussion
Contents organized around indicators• Academic quality• Student-learning
outcomes• Retention and
graduation• Stakeholder input• Program review• Accreditation
22
Some questions you will see• Do we say what and how
much students should learn?
• What kinds of evidence do we collect about student learning?
• Are we benchmarking performance against external standards?
• What progress have we made in addressing recommendations from the last Middle States review?
• Who is responsible for assessment and how it is accomplished?
• How do we use assessment results?
• How does our performance measure up?
• What do student responses tell us about the quality of their academic experiences?
• Are we considering other stakeholder views?
ACADEMIC QUALITY DASHBOARD
Input Indicators
24
Legend: CR = Critical Reading, M = Math, W = Writing
Student Indicators
Faculty Indicators
Mean HS GPA Top 10%
Mean ACT Composite
Mean SAT (CR+M+W)
2010 32.4%
2011 36.8% 2012
35.3%
2010 1757
2011 1785
2012 1800
2010 27
2011 27
2012 28
76.3 76.4
78.9
75.0
76.0
77.0
78.0
79.0
80.0
2010 2011 2012
% of full-time T/TT Instructional Faculty with terminal degrees
9.5 9.6
10.310.5 11.0
10.0
6.0
8.0
10.0
12.0
14.0
2010 2011 2012
% AALANA T/TT Faculty
% A ALANA T/ TT FacultyKRA Goal
31.0
31.6
32.2
31.5
32.0
32.0
30.0
30.5
31.0
31.5
32.0
32.5
2010 2011 2012
% Female T/TT Faculty
% Female T/ TT FacultyKRA Goal
2010 90.0
2011 90.1
2012 90.2
25
Environmental Indicators Student Indicators
1st Yr Persistence
2nd Yr Persistence
Graduation Rate
60 65 70 75 80 85 90 95 100
87.786.3
8988
77.880.7
78.7
66.469.8
67.770
Undergraduate
1st Yr Persistence 2010-112011-122012-13
Graduation Rate 2010-112011-122012-13
60 65 70 75 80 85 90 95 100
76.377.4
76.6
68.973.9
71.5
Graduate
26
Environmental Indicators, cont’d
Student Indicators
NSSE Summary Questions
2006-07 2008-09 2010-111.00
1.75
2.50
3.25
4.00
3.27 3.23 3.27
4-Point Scale (1=Poor and 4 = Excellent)
How would you evaluate your entire educa-tional experience at this institution?
2006-07 2008-09 2010-111.00
1.75
2.50
3.25
4.00
3.03 3.06 3.18
4-Point Scale (1=Definitely No and 4=Probably Yes)
If you could start over, would you go to same institution you are attending now?
27
Environmental Indicators, cont’d Faculty Indicators
*Data source: IPEDS
2010-11 2011-12 2012-130%
20%
40%
60%
80%
100%
49.1% 49.1% 47.5%
66.4%77.7% 74.0%
Section Sizes < 20
Undergraduate Graduate
2010-11 2011-12 2012-130%
10%
20%
30%
40%
50%
60%
21.6% 22.9% 23.7%19.1% 19.5% 20.6%
57.2% 56.6%53.5%
% Sections Taught*
FT Non-TT faculty PT Non-TT faculty T/TT faculty 2010-11 2011-12 2012-130%
1%
2%
3%
4%
5%
3.6%4.1% 4.4%
0.3%
1.2%
0.4%
Section Sizes > 50
Undergraduate Graduate
2010-11 2011-12 2012-130.0%
25.0%
50.0%
75.0%
100.0%95.9% 96.0% 95.9%
Satisfactory = meeting, exceeding, or outstanding
% All FT Faculty Rated 'SATISFACTORY' in Teaching Effectiveness *
*2012-13 Preliminary (does not include NTID). Final available in Nov 2013
2010-11 2011-12 2012-1312.012.513.013.514.014.515.0
13.0 13.0
14.1
Undergraduate + GraduateStudents per Faculty Ratio
*Includes undergraduate + graduate sections
Output Indicators
28
Placement
2009-10 2010-11 2011-12 0%
10%20%30%40%50%60%70%80%90%
100%
20.8%13.8% 11.0%
72.2%77.2% 81.1%
93.0% 91.0% 94.6%
Undergraduate
% Grad School % Employed Total Employed + Grad School
2009-10 2010-11 2011-12 0%
10%20%30%40%50%60%70%80%90%
100%
13.3%6.1% 5.7%
84.4% 85.4% 84.6%
97.7%91.5% 95.1%
Graduate
% Grad School % Employed Total Employed + Grad School
Noteworthy: in 2011-12 HIGH response rate of 85.6%
Output Indicators, cont’d
29
Employer Satisfaction
Learning Outcomes
* 5 point Likert scale, 5 = Excellent and 1 = Poor
92.992.7
91.4
88.0
90.0
92.0
94.0
96.0
2009-10 2010-11 2011-12
Employer Co-op Satisfaction (% would hire permanently)
4.39 4.38 4.39
4.00
4.20
4.40
4.60
4.80
5.00
2009-10 2010-11 2011-12
Overall Quality of Student's Performance*
2009-10
2010-11
2011-12
KRA Goal
0% 20% 40% 60% 80% 100%
40%
56%
75%
65%
*Benchmark Levels = meeting or exceeding
% of All Programs that Achieved Bench-mark Levels*
2009-10
2010-11
2011-12
KRA Goal
0% 20% 40% 60% 80% 100%
90%
80%
84%
85%
% of Programs that Use Assessment Results and Pro-cesses to Guide Planning and Improvement
30
Q05. How well did the highest education from RIT prepare you for each of the following?
Current work status
Getting desired job or into adv degree program right after grad
Further graduate education
Responding to new career opportunities
Deepening my understanding and comm to personal dev
Contributing to my community
PCUAD RIT 2011 RIT 2009
Poor preparation
Fairpreparation
Goodpreparation
Excellentpreparation
*PCUAD = Private College and University Alumni DirectorsAlumni Relations leaders from 40 mid-size to larger-size private colleges and universities (50k-225k alumni) get together twice a year to share the benchmarking data and compare resources and results.
31
Q07. How important was each of the following to your experience as a student, and how well did RIT do at providing them?
Academics and classes
Skills and training for career
Relationship with the faculty
Traditions or values learned on campus
Opportunity to interact with alumni
Attending athletic events
Importance Performance
Not Important
Poor
SomewhatImportant
Fair
Very Important
Good
CriticallyImportantExcellent
RIT 2011
ECC BY-LAWS, 2ND READING
Section 1
Section 2
ECC PLAN OF WORK: 2013-2014
ECC Plan of Work: 2013-2014
Introducing the Board toEducational Quality Oversight: I
• Years of providing information from External Program or Accreditation Review of Academic Programs (most on a 7-year cycle)– Results of most recent year + – 1-year follow up to answer the question: What have we
done to address uncovered weaknesses• No real understanding of what such reviews
entailed– Provided extensive information about how these
reviews are conducted and the role of faculty/staff in each
Introducing the Board toEducational Quality Oversight: II
• Provided extensive detail regarding annual peer review processes related to assessment of student learning outcomes
• Introduced Board to some of the 21st teaching and learning/course redesign efforts in place
• Reported on status of online/hybrid/in-class course delivery
• In-progress efforts to identify University-level graduate learning outcomes
Introducing the Board toEducational Quality Oversight: III• Fall Board Retreat
– Prepared Report – Executive “Summary” (19 pages long) + 22 pages of detailed appendices
– Addressed Peter Ewell’s observations in Making the Grade (2006)
– How MSU Denver• Is embedding High Impact Practices• Adds value to students (modified open enrollment
institution)
– Devoted half-day to discuss
Trustee Jeopardy
Our Students
Graduation/ Pass Rates Our Faculty Our
Programs Metrics
10 10 10 10 10
20 20 20 20 20
30 30 30 30 30
40 40 40 40 40
50 50 50 50 50
10 Point Questions & Answers
• Students– What percentage of MSU Denver students are
transfers?• 60%
• Graduation Pass Rates– True or False: 50% of MSU Denver transfer
students graduate within 7 years?• False: MSU Denver transfer students graduate within 9
years.
10 Point Questions & Answers
• Faculty– How many tenured faculty does MSU Denver
employ? 310 281 223?• 281; MSU Denver also has 179 tenure track
(probationary) faculty members
• Our Programs– What does “SAI” stand for?
• Supplemental Academic Instruction
10 Point Questions & Answers
• Metrics– What % of Latino/Hispanic Enrollment does MSU
Denver need to achieve in order to achieve HIS designation?
• 25% Full Time Equivalent Students (FTES)
What Now?
• Deep focus on retention and how to determine which things we do matter as we seek to enroll, retain, and graduate students
• Academic and Student Affairs Leadership Team (also includes ITS) in constant conversation about what we are doing
• Improving metrics
Board Oversight of Student Learninghttp://agb.org/improving-board-oversight-student-learning
Lessons for Progress1.Ensure sufficient institutional
assessment capability
2.Start with what you already have
3.Make academic quality a priority of the board and institutional leaders
4.Attach the effort to other activities
Peter Eckel, “Lessons Learned about Student Learning: 8 Test Cases,” Trusteeship, Jan-Feb 2014
Lessons for Progress5. Educate the board on
education
6. Find the right focus
7. Allow for targeted deeper dives
8. Develop new board processes and use time differently
9. Deepen the board’s engagement with faculty
Peter Eckel, “Lessons Learned about Student Learning: 8 Test Cases,” Trusteeship, Jan-Feb 2014
Thank you Peter Eckel ([email protected])Vicki Golich ([email protected])Jeremy Haefner ([email protected])