What the *!# Is Middle States What the *!# Is Middle States Looking For?Looking For?

A Review of Standards 7 and A Review of Standards 7 and 14—Institutional & Student 14—Institutional & Student

Learning AssessmentLearning Assessment

Presentation to the Association of Institutional

Research and Planning OfficersAlbany, New York

June 17, 2010

PresentersPresentersPatty Francis

Associate ProvostInstitutional Assessment & Effectiveness

SUNY Oneonta

Sean McKitrickAssistant Provost

Institutional Research & AssessmentBinghamton University

Session ObjectivesSession ObjectivesTo examine MSCHE’s expectations for

Standards 7 and 14To describe strategies campuses might

use in preparing to meet those standards (and convince Middle States they meet them!)

Background Background InformationInformation

Recent Changes at MSCHERecent Changes at MSCHEStrengthening of standards in 2002

◦Emphasis on importance of student learning outcomes, reflected in creation of new standard for student learning (i.e., Standard 14)

◦New focus on “institutional assessment” (i.e., Standard 7)

Addition of “assessment expert” to evaluation teams and hiring of Linda Suskie, nationally-known figure in learning outcomes assessment

Even more changes as a result of national pressures (e.g., Spellings Commission, HEA)

MSCHE Expectations for MSCHE Expectations for Planning and AssessmentPlanning and Assessment

“It is the Commission’s intent, through the self-study process, to prompt institutions to reflect on those assessment activities currently in place (both for institutional effectiveness and student learning), to consider how these assessment activities inform institutional planning, and to determine how to improve the effectiveness and integration of planning and assessment.”

Range of Middle States ActionsRange of Middle States ActionsReaffirm accreditation (that’s the best one!)Reaffirm accreditation and request that PRR

address specific issues that need attentionReaffirm accreditation and request a progress

letterReaffirm accreditation and request a monitoring

report (team visit may follow)Warn institution accreditation may be in jeopardy

and request monitoring report (visit will follow)Postpone decision on accreditation and request

supplemental information report

Is Middle States Serious?Is Middle States Serious?• Review of MSCHE Website for SUNY

institutions, March 2005 to November 2009– For 10-year accreditation:

• No assessment issues for 13 institutions• Progress letters requested for 3 institutions • Monitoring reports requested for 7 institutions• Warnings issued for 2 institutions

– For PRR:• No assessment issues for 19 institutions• Progress letters requested for 9 institutions • Monitoring reports requested for 3 institutions• Warning issued for 1 institution• Accreditation decision postponed for 1 institution

Feedback Session #1: How Feedback Session #1: How Ready is Ready is YOURYOUR Campus? Campus?

• When is your next accreditation action (10-year, PRR)?

• Were you required to provide follow-up after last action related to assessment?

• Do you have specific concerns related to:• Standard 7?• Standard 14?

Understanding Understanding Standard 14Standard 14

and Preparing for It

Fundamental Elements for Fundamental Elements for Standard 14Standard 14

• Clearly articulated statements of expected student learning outcomes at all levels that are integrated and congruent with institutional mission as well as with standards of higher education and of discipline

• Use of assessment process that is organized, documented, and sustained

Fundamental Elements for Fundamental Elements for Standard 14 (cont.)Standard 14 (cont.)

• Use of measures that:– Are direct– Consist of multiple qualitative and/or

quantitative indicators– Maximize existing data and information– Relate clearly and purposefully to the goals

they are assessing• Support and collaboration of faculty and

administration is apparent• Existence of clear, realistic guidelines

and timetable as well as supported by institutional resources

Fundamental Elements for Fundamental Elements for Standard 14 (cont.)Standard 14 (cont.)

• Provisions for evaluating and revising the assessment process itself

• Evidence that students are achieving key institutional and program outcomes

• Mechanisms for sharing and discussing assessment results and using them to improve teaching and learning

• Documented use of assessment results as part of institutional assessment (link to Standard 7)

Issues of Particular Interest Issues of Particular Interest to a Reviewerto a Reviewer

• Distinction between accredited and non-accredited programs

• Are “ordinary” faculty (i.e., not chairs or others in leadership positions) involved in assessment?

• Are programs engaging in PROGRAM assessment (i.e., as opposed to COURSE assessment)? [Note: Don’t forget gen ed!]

• Is the institution supporting assessment, and what’s the proof?

• What happens to assessment data after measures are administered?

Understanding Understanding Standard 7Standard 7

and Preparing for It

Feedback Session #2: What’s Feedback Session #2: What’s Your Reaction?Your Reaction?

What is absurd about this commercial?What does it say about the relationship

between quality and planning?What does it say about the relationship

between assessment and quality?

Fundamental Elements Fundamental Elements for Standard 7for Standard 7

• Documented, organized, and sustained assessment process to evaluate and improve the total range of programs and services and achievement of institutional mission, goals, and plans

• Written institutional (strategic) plan that reflects consideration of assessment results

• Evidence that assessment results are shared and used in institutional planning, resource allocation, and renewal

Background IssuesBackground Issues

Relationship between Standards 7 and 2Does not obviate need for addressing

issues relating to Standard 14Key words—planned, sustained,

strategic, useful, meaningful, cost effective

From: Suskie, L. 2009. :understanding Middle States Expectations for Assessment, 2009 AIRPO Conference, Buffalo, NY.

Issues of Particular Interest Issues of Particular Interest to a Reviewerto a Reviewer

Student learning goes beyond the classroom, so where is this assessed?

If unit focuses on students, this needs to be assessed

Assessment in non-academic or administrative units (from facilities to the bursar)

Coordination of assessment processes

Importance of Overall Importance of Overall Strategic PlanStrategic Plan

Mission and visionGoals/ObjectivesPerformance indicatorsAlignment between overall plan and “the

trenches”Periodic (not episodic) use of assessment

data for budgeting and decision-making

Two Tough Nuts to Two Tough Nuts to CrackCrack

Documentation and Assessment as “Sustained” Activity

How Do We Document How Do We Document Processes?Processes?

Meeting minutesData dashboardsUnit assessment plansUnit narrativesUnit/division/university strategic plans

with annual reports

Ways to Assess

Ways Assessments

are Used

Ways to Document

Impact

Community Service

Internship Survey

Discussed in senior staff meetings

Meeting minutes; curriculum docs

Productivity Bi-annual productivity studies

Discussed with deans and dept heads

Revenue

Access

Diversity

Scholarship

Learning

Assessment, Use, and Assessment, Use, and DocumentationDocumentation

Documenting Linkage Between Budgeting, Documenting Linkage Between Budgeting, Assessment, & Strategic PlanAssessment, & Strategic Plan

Graphing ResultsGraphing Results

How Do We Demonstrate How Do We Demonstrate Sustained Assessment?Sustained Assessment?

Keep good records of all assessment activity, preferably in a central repository

Keep (and update regularly) a “history of assessment” on your campus

Be able to demonstrate your campus has responded appropriately to past MSCHE suggestions/recommendations

Perhaps most important, practice “assessment for improvement”

Assessment for Assessment for ImprovementImprovement

Assessment for Assessment for AccountabilityAccountability

1. Major impetus: Using data to get better

2. Process is ongoing, part of “SOP”

3. Process is faculty/ staff driven (with support)

4. Assessment seen as valuable – and valued – by institution

1. Major impetus: Satisfying accreditors

2. Process “stop and start,” intermittent

3. Process is “top-down”

4. Institution pays little attention to assessment (once accreditors are gone

From Banta, T.W. (2007). Can assessment for accountability complement assessment for improvement? Peer Review, 10-12.

What the *!# Is Middle States What the *!# Is Middle States Looking For?Looking For?

A Review of Standards 7 and A Review of Standards 7 and 14—Institutional & Student 14—Institutional & Student

Learning AssessmentLearning Assessment

Presentation to the Association of Institutional

Research and Planning OfficersAlbany, New York

June 17, 2010