NCSD National Conference Denver, CO Oct. 22-24, 2009

Elise Davis-McFarland, Vice President

for Student Services

Sharon Kearns, M.Ed., Director of Testing

We Are All Accountable: Student Learning Outcomes for Student Services

Student Affairs professionals have the responsibility for ensuring that

institutions of higher education become true learning communities committed

to providing transformative educational experiences for all students.

Learning Reconsidered – ACPA, NASPA

The institution provides student support programs, services, and activities consistent

with its mission that promote student learning and enhance the development of its

students. (Student Support Services)

Norht North Central Association Commission on Accreditation and School Improvement

Learning goals are designed to press all students to excellence and focus on enhancing the intellectual, personal, physical, social, and career development of students.

PS 1 The learning goals are challenging, are appropriate in terms of rigor and diversity, meet student needs, and are consistent with the stated goals of the program.

PS 2 Objectives are: identified for the specific subject areas and the various programs such as the activity programs, and student personnel services;

Different Outcomes

Student service Satisfaction, needs, program quality

Student development Values, attitudes, beliefs, psychosocial

development Learning

Knowledge and skills

Why Student Learning Outcomes?

The business of colleges is teaching and learning. If you’re not a player, you’re just sitting on the sidelines.

.

The actual process ismuchlike….

First ……you must lay the groundwork…

You need a frame of reference

1. Learning Reconsidered - ACPA/NASPA Cognitive complexity Knowledge, acquisition, integration,

application Humanitarianism Civic engagement Interpersonal and intrapersonal competence Practical competence Persistence and academic achievement

You need a frame of reference

2. Principles of Good Practice for Student Affairs- ACPA/NASPA

Active learning Values & ethical standards High expectations for student learning Systematic inquiry Effective resource use Educational partnerships Community

Assessment should show that Student Services

Plays a significant role in student learning

Makes a difference in academic success

Promotes retention and persistence

Enhances cognitive development

Enhances affective development

Brainstorming & Consensus

How about?How about…?

Hmm..

Hmm..

What if we…?

What if a student asks…

What should I be able to do as a result of my interaction with the______ office in Student Services?

What will I learn from meetings with my financial aid officer?

What would you say?What do you expect students to be able to

do/know as a result of their interactions or participation

What should students gain from Student Services?

Essential life skills Personal responsibility Perseverance Decision making/critical thinking Financial responsibility Technology use

What should students gain from Student Services?

Essential social skills Effective communication Interpersonal skills Civic mindedness

Ethical behavior Adherence to legal and ethical

requirements

Davis-McFarland, E.

Planning Pyramid

Student Affairs Divisional Plan

College Strategic Plan

Institutional Mission

Adapted from Bentrim-Tapio, 2008

Mission Pyramid

Unit Mission (EOP, EM, SD)

Division

Mission

Institutional Mission

Mission Statements: Before As part of the Student Services Division of Trident Technical College, the staff members of the Financial Aid/Veterans Assistance Office are committed…….

Trident Technical College Registrar’s office mission is to provide the best possible service…..

The Educational Talent Search program will sustain a supportive and……

The mission of Career and Employment Services is to prepare students for the transition from……

Student Activities’ mission is to complement the academic program

and enhance the students’ college….

What’s missing?

Davis-McFarland, E.

BEFORE Mission Statement Example

Orientation ServicesOur Purpose...

The mission of Orientation Services at Trident Technical College is to provide a welcoming environment in which new students and their families obtain information necessary for a successful transition into college life. Through our orientation program, we encourage campus involvement, academic success, and student retention. Our aim is to integrate new students and families into campus life while easing the transitions and stresses associated with starting college. In order to achieve these objectives, students and families who attend an

orientation will…..View a video to familiarize them with the campus environment and physical

facilitiesExplore Trident Technical College’s Website Talk with an Orientation Leader to

Verify the student’s major Learn about important processes, like registration Be assigned an academic advisor Receive important printed materials about the CollegeFind out about student services and organizations Have questions answeredTake a walking tour of the campus, if desired Register for courses during the registration period in the Orientation Services

Centers if they are non-degree students

AFTER

Orientation Services

In support of the Student Services Divisionmission of encouraging student success, Orientation Services provides acomprehensive orientation program toencourage campus involvement, academicsuccess, and student retention.

A good learning outcome

Adapted from Bentrim-Tapio, 2008

Indicates what the student should know or be able to do

Measurable

Related to experiences you provide

Realistic

Aligns with vision, mission and values

Adapted from Bentrim-Tapio, 2008

Learning Outcome Statements

Specify what students will be able to do, know, and/or demonstrate as a result of participating in or completing an activity/service/program.

Should be expressed as knowledge, skills, attitudes or behaviors.

We always use verbs

Bloom’s Taxonomy Describe Demonstrate Develop Identify

Difficult to Measure Appreciate Aware of Familiar with

The wording….As a

result of

BEHAVIOR• Meeting• Participa

ting• Working

with ACTIVITY• Program• Service• Activity

Who• Students• Athletes• Officers

Active Verb• Demonst

rate• Identify• Explain

EXPECTATION • 50%

improvement• Average

performance• Pre-test post-

test improvement

INTENTION Mock interview• Time

management

Adapted from Bentrim-Tapio

Learning Outcome Examples

As a result of students attending financial aid workshops and receiving financial aid education, the student default rate will not exceed 5%.

As a result of instruction from the test proctor, students will manage their time in order to complete their tests within the allotted time.

As a result of attending officer training sessions and holding office, organization officers will demonstrate improvement in their leadership skills from the beginning of their tenure through completion.

The Assessment Cycle

Groundwork

Consensus

Form SLO Committee Mission

ObjectivesGoals

Outcomes

Revise mission statements

Develop surveys,forms, rubrics, etc.

Collect data(develop schedule)

Analyze & reportfindings

Determine findingsuse to enhance & improve

Satisfaction vs. Learning is much like the relationship between Reliability and Validity.

Reliability means nothing when the problem is validity, satisfaction isn’t too important if students don’t learn.

Satisfaction surveys

Bresciani, 2002

don’t help you understand all of the contributions of your programs & services

don’t tell you how your programs and services contribute to student development & learning

don’t tell you everything you need to know about programs & service improvements

Adapted from Bresciani, 2002

Satisfaction

85% of students will agree or strongly agree that mock interviews helped them feel more confident in their interviewing skills.

Sir, we’ve had a paradigm shift since you left.

Learning

85% of students who attend the How to Interview workshop will score average or above average on a mock interview evaluation matrix.

Bresciani, 2002

Student Services Learning Outcomes

Shows Academics that Student Services makes significant contributions to learning

Forms the basis for collaborations that

Promote measurable student learning Promote student engagement and socio-academic

integration Provides models for indirect learning

Examples

Davis-McFarland, E.

2006-2008 Student Services Program Assessment

Department: Orientation Services Director: Brandon Branham Division: Student Services Dean: John Jamrogowicz

Assessment Summary

Key Findings Plan for Improvement Implementation The assessment provided information on student learning outcomes in the areas of advising session preparation, TTC website knowledge, and information retention. Areas of strength for the department are in advising session preparation. Information retention and website knowledge require follow-up.

The department will work on improving students’ website knowledge and on gathering accurate results for information retention in orientation sessions.

Orientation Leaders are trained to instruct students on the importance of website knowledge.

The Student Services Division encourages student success by providing programs and services that support students’ accomplishment of their academic, student leadership, and social development goals. All student services programs, initiatives, and services are proved in keeping with the principles of good practice for student affairs and services. The Enrollment Management Unit provides an organizational framework and leadership that facilitates cohesion and coordination between and among its component admissions, financial aid, orientation, registrar, and testing services. We intend to be reliable and effective partners in helping our faculty and fellow staff members accomplish their work with our students from entry to exit. We will accomplish our work in a manner that causes other state and regional two-year colleges to recognize us as an enrollment management unit worthy of benchmarking. Orientation Services provides a comprehensive orientation program that assists students in achieving both their academic and developmental goals through a comprehensive orientation program. We encourage campus involvement, academic success, and student retention. Our aim is to integrate new students into campus life while easing the transitions and stresses associated with starting college.

Davis-McFarland, E.

STUDENT LEARNING OUTCOMES FOR PROGRAM REVIEW Department: Orientation Services

Division: Student Services Year: 2006- 2007

A. Student Learning Outcomes

B. Where Outcomes are Assessed

C. Methods for Outcomes Assessment

D. Expected Level of Program Performance

E. Data Collection F. Results G. Plan For Improvement

What should students who interact with your service area be able to do? (1 outcome per block)

Where do you see evidence that the student can do these things?

How does your program evaluate student/graduate skills/abilities?

What is the expected level of student performance for the program?

How and when will you collect the data needed to evaluate the performance of the program?

What are the results of the evaluation?

How will you use this information to improve the program

Students demonstrate an ability to adequately prepare for an academic advising session.

Created new student matrix to be used by Orientation Leaders for pre/post-test data collection regarding advisor visits.

Matrix administered to students prior to and after individual orientation sessions. (pre/post-test)

> 80% of the students will correctly answer post-test questions. Post-test scores will improve ≥20%.

Review of pre/post-test scores from student matrix administered October 2007 - November 2007.

100% of students correctly answered post-test questions

Actions: 1. During orientation sessions,

emphasis will be placed more heavily upon the importance of preparing adequately for an advising session.

Eliminated student matrix for next cycle. Will use original matrix but will speak directly to Advisors and not to students. Percentages were met or exceeded for Fall 2006. No change is indicated.

Advisor Feedback

Advisee evaluation matrix

≥60% of students have acceptable evaluations

Review of advisee evaluation matrices completed by 10 faculty advisors for Priority Registration for Fall 2006 term.

75% of advisees called or e-mailed to set up an appointment. 69% of advisees brought necessary documents to their advising sessions. 95% of advisees communicated course requirements or preferences. 66% of advisees arrived on time for advising sessions.

Davis-McFarland, E.

PROGRAM VITAL STATISTICS

Indicator Trend Analysis

Orientation Numbers by Calendar Year

Mean of 2274 and standard deviation of 211 Trend analysis indicates a very strong decreasing linear trend in the number of Orientations per year. The reported p-value is 0.003

0

500

1000

1500

2000

2500

3000

2003 2004 2005 2006 2007

25622416

2204 2149 2039

Davis-McFarland, E.

2007-2008 Student Services Program Assessment

Department: Financial Aid Director: Ellen Green Division: Student Services Dean: John Jamrogowicz Number of awards FY 2008: 37,189 Total amount awarded for FY 2008: $43,522,825 Default Rate: 5.5%

Assessment Summary

Key Findings Plan for Improvement Implementation The assessment provided information on student learning outcomes in the areas of student’s financial responsibility and demonstrates an understanding of the importance of academic progress. Areas of strength are in the financial responsibility of students who are affected by Return to Title IV (R2T4). The data indicates the need for additional resources regarding loan eligibility and borrowing responsibilities.

Investigate the hiring of a loan default coordinator to work with data from the National Student Loan Data System (NSLDS) to educate students on their financial rights and responsibilities regarding loan borrowing. Review the data to determine if the expected level of performance should be adjusted in light of the expanded loan limits from the Department of Education.

Requested and received approval for the position and hired a Loan Default Coordinator. The coordinator is using resources from NSLDS, ELM, and national data to provide students with information regarding Financial Aid rights and responsibilities in addition to how to borrow responsibly. The department is offering workshops both in classes and during day/evening hours to explain both the financial responsibilities of students and the importance of academic progress for continued eligibility for financial aid.

The Student Services Division encourages student success by providing programs and services that support students’ accomplishment of their academic, student leadership, and social development goals. All student services programs, initiatives, and services are proved in keeping with the principles of good practice for student affairs and services.

The Enrollment Management Unit provides an organizational framework and leadership that facilitates cohesion and coordination between and among its component admissions, financial aid, orientation, registrar, and testing services. We intend to be reliable and effective partners in helping our faculty and fellow staff members accomplish their work with our students from entry to exit. We will accomplish our work in a manner that causes other state and regional two-year colleges to recognize us as an enrollment management unit worthy of benchmarking.

Davis-McFarland, E.

STUDENT LEARNING OUTCOMES FOR PROGRAM REVIEW Department: Financial Aid

Division: Student Services Year: 2007 - 2008

A. Student Learning Outcomes

B. Where Outcomes are Assessed

C. Methods for Outcomes Assessment

D. Expected Level of Program Performance

E. Data Collection F. Results G. Plan For Improvement

What should students who interact with your service area be able to do?

Where do you see evidence that the student can do these things?

How does your program evaluate student/graduate skills/abilities?

What is the expected level of student performance for the program?

How and when will you collect the data needed to evaluate the performance of the program?

What are the results of the evaluation?

How will you use this information to improve the program

Students receiving financial aid demonstrate financial responsibility.

Default Rates Review default rates The loan default rate for the college is ≤5%.

Review of Department of Education default rate, March 2007.

5.5% of TTC borrowers are in default.

Actions: 1. More aggressively contact

students regarding their status

2. Utilize national data to track students

3. Update data on website to help with understanding borrowing rights and responsibilities

Loan Exit Counseling

Students successfully complete loan exit counseling.

≥85% of students successfully complete loan exit counseling within 2 months of notification.

Students who received loan exit counseling notifications during the Fall 2007 term completed the counseling by 2/28/08.

3.5% of students completed required counseling within 2 months

Actions: 1. Increase access to national

data. 2. Notify students earlier of exit

requirements 3. Provide direct contact with

students when they apply for graduation

Borrowing Rates Review average student loan debt of the graduating class.

Average loan debt for the graduating class does not exceed $5250. (Note: The $5250 was based on average four-year eligibility for loan)

Review Datatel reports which identify the average loan debt for the Spring 2008 graduates

Average borrowing per graduate is $6898

Actions: 1. Review and adjust average

loan borrowing rates due to changes in maximum limits.

2. Review rules regarding packaging of student loans.

3. Conduct individualized counseling to discourage excessive borrowing

Davis-McFarland, E.

VITAL STATISTICS

Indicator Trend Analysis

Loans: Trend analysis indicates a strong increasing linear trend with a p-value of 0.015. The total loan amount has increased by 86.6% since 2004. Grants: Trend analysis indicates a rebounding cubic trend with a p-value of 0.10. The total grant amount has decreased by 4.7% since 2004. Lottery: Trend analysis indicates a weak decreasing quadratic trend with a p-value of 0.22. The total lottery amount has increased by 5.6% since 2004. Scholarship: Trend analysis indicates a strong increasing linear trend with a p-value of 0.01. The total scholarship amount has increased by 59.4%. Work-Study: Trend analysis indicates a weak increasing linear trend with a p-value of 0.16. The total work-study amount has increased by 14.3%.

$-

$5,000,000

$10,000,000

$15,000,000

$20,000,000

2004 2005 2006 2007 2008Loan $11,662,833 $13,454,167 $19,584,212 $19,153,743 $21,774,304

Grant $13,618,677 $13,467,035 $12,770,029 $12,100,065 $12,977,851

Lottery $6,900,605 $7,426,861 $7,553,574 $8,182,811 $7,292,341

Scholarship $677,768 $654,815 $802,125 $953,854 $1,080,503

Work-Study $348,031 $376,733 $400,213 $449,859 $397,826

Total Financial Aid Disbursed by Award Type(by fiscal year ending June 30)

Source: TTC Office of Finance and Administration: Comprehensive Annual Financial Report June 30, 2008Updated: March 3, 2009