Utilizing Peer Leadership Toward a Systemic

Approach to Student Learning Outcomes

Design

Brett Perozzi Dianna Rangel

Prasanna Reddy

Weber State University

Introduction

• Presenters• Overall Context and Mandates• Evolution of the WSU Process• Current Status• Critical Elements• Administrative Considerations• Moving Forward• Q & A.

Context for Presentation

• Accreditation-oriented• Emanated from uncertainty

– No training, knowledge base, etc.

• Organic process• A culture of collaboration• Web interface to assist process

Mandates and Expectations• Calls for increased accountability –

1970s• Public decline of confidence in

American higher education• Escalating costs in both private and

public institutions• Burgeoning questions about the return

on investment• Most recently, Spellings Commission.

Ahumada, 1986; Romer, 1995; Wingspread report, 1993; Leslie & Brinkman, 1988

In the Beginning…

• Ideal WSU Baccalaureate Graduate Task Force (AY 05-06)– Based on campus-wide conversations re:

qualities of the “ideal WSU baccalaureate graduate”

• Communication• Working with Information• Basic Knowledge of Core Disciplines• Social and Cultural Factors

In the Beginning…

– Charge: How does Student Affairs contribute to these qualities?

• Task force members randomly assigned by VPSA

• Uncertainty re: objective• Backward assessment approach

– Retrospective– Unintentional

A Focused Transition

• New VPSA Feb 2006• From Ideal Grad Task Force to Student

Learning Outcomes Task Force (SLOTF)– Focus on intentionality– Know / do / believe– Learning Reconsidered/2

A Focused Transition

• Counseling• Testing• Tutoring• Career• Disability

Services

• Housing• Technology• Student Union• Faculty• Associate VPSA

• Broadened committee

• Diverse in levels and areas:

• New Mission Statement

SLOTF Mission Statement• The Student Learning Outcomes Task Force

(SLOTF) will educate the Division of Student Affairs about the concept of student learning outcomes in higher education. Relevant speakers, in-service training opportunities, and readings will be utilized to promote a paradigm shift throughout the Division. A small number of specific student learning outcomes will be identified as particularly pertinent within the Division. SLOTF will support departments as they set goals and assess these student learning outcomes, and will provide implementation assistance and consultation as needed. National standards and tools, local benchmarking, and pilot testing will informthis process.

Review of Theory and Best Practice

• Learning Reconsidered 2• WSU Documents

– General Education Mission & Goals– Board of Regents System Goals– Ideal WSU Baccalaureate Graduate

Qualities

Review of Theory and Best Practice

• Best practices referenced in Learning Reconsidered 2– Westminster College– University of Minnesota– Longwood University– Spokane Community College– California State University System– Bridgewater State College

Just Do It• Preparing for “round one” (06-07 SLOs)

– Professional Consultations• Local expert on assessment basics (Dr. Kari

Ellingson, Oct. 2006)• National expert on student learning outcomes

and outcomes-based assessment (Marilee Bresciani, Dec. 2006 and Dec. 2007)

– Encouragement to build WSU model of student learning outcomes organically

– Start with one small and measurable outcomes-based assessment project per department

Just Do It• Submission deadlines supported by

VPSA• SLOTF assistance:

– Web resources– Open consultation times– Forms

Assessment Plan MatrixName of Program:

Date:

Program goal/objective

Student learning outcome associated with the program goal/objective identified above(indicate what students will be able to know, think, and/or do; use detailed, specific, and measurable language)

Program activity that is designed to teach students the learning outcome identified above

Assessment strategy that will measure student learning outcome identified above

Assessment Plan MatrixName of Program: Testing Centers - Academic Support Centers and Programs

Date: January 16th, 2007

Program goal/objective All proctors must be trained in the policies and procedures concerning the testing center.

Student learning outcome associated with the program goal/objective identified above(indicate what students will be able to know, think, and/or do; use detailed, specific, and measurable language)

Proctors will demonstrate their knowledge and practice of all testing procedures and policies.

Program activity that is designed to teach students the learning outcome identified above

1.One-on-one training at the time of hire2.Hands-on training first week of employment

Assessment strategy that will measure student learning outcome identified above

1.Test on policies and procedures2.Supervisor observation

Assessment Plan MatrixName of Program: WRC Staff Professional Development

Date: 2008

Program goal/objectiveWell trained staff will be better prepared to interact with WRC customers when afforded the opportunity to gain hard skills and personal experience in activities related to WRC operations.

Student learning outcome associated with the program goal/objective identified above(indicate what students will be able to know, think, and/or do; use detailed, specific, and measurable language)

1) Staff will be know and demonstrate proficiency in three main areas related to the WRC rental center operations. Staff will be able to demonstrate proficiency in ski package set up, snowboard package set up, and waxing/tuning of skis and snowboards.

Program activity that is designed to teach students the learning outcome identified above

1) Staff will participate in a two part professional development session during the last week in March. 2) Staff will receive a recap on hard skills used throughout the winter in our rental shop. Skills will include ski package set up, snowboard package set up, and waxing/tuning.

Assessment strategy that will measure student learning outcome identified above

1) During the second part of our two part professional development session during the last week in March, staff will demonstrate their proficiency to properly set up a ski package, a snowboard package, and wax/tune a ski or snowboard.2) Effectiveness will be measured based on the outcomes of the two packages set up by the staff, and the tuning exercises.3) Material for assessment will be based on criteria set forth in the first professional development session. 4) Criteria will include:Was the ski/snowboard package set up properly to the size, height, weight and ability of the customer? Was the wax/tune preformed properly? Is there sufficient wax on the ski or board?

Assessment Plan MatrixName of Program: Student Leadership Outcome

Date: 2008

Program goal/objectiveSuccessful communication between Student Council members and the campus/community who attend events to foster inclusive feelings and provide information.

Student learning outcome associated with the program goal/objective identified above(indicate what students will be able to know, think, and/or do; use detailed, specific, and measurable language)

Student Council members will articulate and demonstrate the communication principles of opening and closing an event.

OPENING1. Welcome and thank people for coming2. Introduce yourself and the other members of the Council3. Remind the audience of upcoming events4. Hand out any appropriate flyers or brochures5. Provide any instruction/information pertinent to the event

CLOSING1.Thank people for coming again2. Remind again of upcoming events3. Welcome suggestions for events or improvements

Program activity that is designed to teach students the learning outcome identified above

1. Training sessions by advisors at Student Council Winter Retreat2. Demonstration at events

Assessment Plan MatrixName of Program: WRC Staff Professional Development

Date: 2008

Assessment strategy that will measure student learning outcome identified above

1. Written assessment of principles2. Advisor observation and evaluation3. Self-evaluation

ASSESSMENT MEASUREMENTScale of 1 to 41 = Does not perform the function2 = Performs the function, but in a disorganized and incomplete manner3 = Performs the function in an organized OR complete manner4 = Performs the function in an organized AND complete manner

Building the WSU Model Organically

• SLO Card Sort Activity– Borrowed from psychological and career

counseling– Envisioned a factor-analysis-like outcome

• What outcomes naturally cluster together?• What should that cluster be named?

– Each outcome placed on a numbered card– “Sanitized” outcomes where possible

• Promote grouping by outcome theme, not source area within division

Card Sort with Management• 25 Learning Outcomes• Sort into Categories• Begin building the model• Foster division-wide collaboration on SLO

initiative• Generate buy-in from all division member• Have fun with outcomes-based assessment• 5 Groups of Staff Members

– Davis Campus

– Student Life

– Student Union

– Dean of Students

– Academic Support Services and Programs

Building the WSU Model Organically

Categories that emerged from “round one”:

• Responsibility & Accountability (14)• Interpersonal Communication (13)• Intrapersonal Competence (6)• Leadership & Management (4)• Critical Thinking (3)• Cultural Competence (1)• Civic Engagement (1)

2006-07 STUDENT LEARNING OUTCOMES

REPORT OF ASSESSMENT RESULTS (7/27/07 version)

Directions: Assessment activities help you improve your programs and

services; they are not an end in themselves. Your report of assessment results

is one way for you to document that you have “closed the loop”, and have

used your assessment data to improve your programs and services. Please

keep this report brief so it does not require a lot of effort for you to complete.

In your report, answer each of the following questions.

Name of office submitting the report:

1. What student learning outcomes were assessed in 2006-07? (State the learning outcomes.)

2. How did you assess these outcomes? (Identify the assessment strategies you used.)

3. What were the results of your assessment? (Summarize the results in ways that will be meaningful and clear to someone outside your office.)

4. What changes will you make based on these results? (Describe how you will change your programs or services, your assessment strategies, your learning outcomes, etc.)

STUDENT LEARNING OUTCOMES - DIVISION OF STUDENT AFFAIRS

REPORT OF ASSESSMENT RESULTS – REVIEW QUESTIONSDepartment :

ASSESSMENT RESULTS PROMPTSAND REVIEW QUESTIONS

YES NO MAYBE

COMMENTS

How did you assess the student learning outcomes?(Identify the assessment strategies you used.)

1. Do the means of assessment measure the student learning outcomes?

2. Do the means of assessment seem feasible and appropriate, given the available time and resources?

3. Are multiple means of assessment used, in most cases?

What were the results of your assessment?(Summarize the results in ways that will be meaningful and clear to someone outside your office.)

3. Are “criteria for success” established for each means of assessment (what level of performance is acceptable)?

3. Are sufficient data provided in the Report of Assessment Results to convince the reader that the assessment described actually took place?

3. Are the assessment results analyzed in such a way as to focus on the accomplishment of the intended student learning outcomes?

What changes will you make based on these results?(Describe how you will change your programs or services, your assessment strategies, your learning outcomes, etc.)

3. Is there evidence of broad-scale staff involvement in the use or discussion of the results?

3. Does the proposed use of the results relate to the intended outcome, and does the use of the results seem reasonable?

STUDENT LEARNING OUTCOMES - DIVISION OF STUDENT AFFAIRS

REPORT OF ASSESSMENT RESULTS – REVIEW QUESTIONS

ASSESSMENT RESULTS PROMPTSAND REVIEW QUESTIONS

YES NO MAYBE

COMMENTS

How did you assess the student learning outcomes?(Identify the assessment strategies you used.)

1. Do the means of assessment measure the student learning outcomes?

1 X 2 X

2. Do the means of assessment seem feasible and appropriate, given the available time and resources?

1 X 2 X

3. Are multiple means of assessment used, in most cases?

1 X 2 X

What were the results of your assessment?(Summarize the results in ways that will be meaningful and clear to someone outside your office.)

3. Are “criteria for success” established for each means of assessment (what level of performance is acceptable)?

1X 2 X

Data not reported.

3. Are sufficient data provided in the Report of Assessment Results to convince the reader that the assessment described actually took place?

1 X2 X

3. Are the assessment results analyzed in such a way as to focus on the accomplishment of the intended student learning outcomes?

1 X2 X

What changes will you make based on these results?(Describe how you will change your programs or services, your assessment strategies, your learning outcomes, etc.)

3. Is there evidence of broad-scale staff involvement in the use or discussion of the results?

1 X 2 X

Not clear which outcome is being discussed. Election versus collaboration.

3. Does the proposed use of the results relate to the intended outcome, and does the use of the results seem reasonable?

1 X 2 X

More clarity is needed.

Current Status

• Web Resources: http://saweb.weber.edu/newsaweb/default.aspx

• Second round of Learning Outcomes submitted

• Continued assessment of first round by individual departments

• New Assessment Coordinator on board

• Assessment of the role of SLOTF

Potential Barriers

• Working almost exclusively from the ground up can be a dichotomy– On the one hand, it is empowering and engenders

greater buy-in– On the other hand it is far more work and may not hit

the target administratively

• Bringing the division directors/leaders up to the same level of understanding

• Unsure which areas (e.g., units within depts.) should be submitting outcomes.

Supports

• Empowering the leaders– Particularly relevant when peer-lead

• Providing education (Ideal Graduate Task Force example)

• Helping with training/pro dev, deadlines, etc.• Technology

– Great people!– Regular members of the committee– Lots of work to build and maintain

• Reshaping division structures, processes, etc. to align/ reinforce values– Recognition, strategic planning, etc.

Fiscal Implications

• Staff time– Training/development– Meetings– Conceptualizing, writing, submitting,

evaluating SLOs, etc.

• Training and development• Consultants, conferences, etc.• Awards/rewards.

Moving Forward

• Tying outcomes based assessment to a broader context– We moved quickly into SLOs and don’t have

a fully developed context for assessment• Hired an assessment coordinator!• Assessment Pyramid concept• SLO tie to overarching initiatives

– Strategic planning, direction, vision, priorities, initiatives, goals, timing, budgets/funding, etc.

Questions

and

Answers