Setting Program and Learning OutcomesCarteret Community College

Learning Outcomes Workshop

June18-20, 2007

We Are Not Doing This for SACS SACS is our accreditation agency They value outcome assessment – but why? Good institutions are concerned with the

following things: Student learning in the classroom Student experiences at the institution Being good stewards of our resources Running effective and efficient institutions

At the end of the day, we want to be able to say… Our students are learning to the best of their ability We are doing an effective job of teaching Students are better off once they leave us than they

were when they came to us The experiences they had in student services were

beneficial They are achieving exactly what we expected them

to achieve We have not placed barriers in front of them

What Else Would We Like To Say? Are we serving our community well? Are we meeting our mission? What else?

Let’s Review

Why Outcomes? Outcomes are program-specific They measure the effect of classroom activities

and services provided. Outcomes represent a new way of thinking Outcomes have become widely accepted by our

various publics They are here to stay We used to measure ourselves by our activities

Outcomes are ……...

Driven by the mission. Related to overall program goals. Specific to the teachings/activities of your

program/course. Determined by faculty and front-line staff. Measured carefully and specifically.

Different Types of Outcomes

Learning Outcomes (can be at course, program or institutional level)

Program Outcomes Administrative Outcomes

Definitions and Examples Learning Outcomes:

What changes in knowledge, skills, attitude, awareness, condition, position (etc.) occur as a result of the learning that takes place in the classroom. These are direct benefits to students.

Examples: general learning skills (e.g. improved writing and speaking abilities), ability to apply learning to the work environment (e.g. demonstrate skills in co-op), program-specific skills developed or enhanced (e.g. take blood pressure.)

Definitions and Examples Program Outcomes:

The benefits that results from the completion of an entire program or series of courses. Are there benefits for students who get the AAS in welding versus those who take a few courses? If so what are they?

Typical examples are: licensure pass rates, employment rates, acceptance into 4-year schools, lifelong learning issues, contributions to society, the profession, etc.)

Definitions and Examples Administrative Outcomes

Units/programs want to improve services or approach an old problem in a new way.

They want to become more efficient and effective. Typical examples are:

All faculty will attend one professional meeting annually so they can stay up-to-date in their field, or:

What is an Outcome Objective?

A short-term, measurable, specific activity having a time limit or timeline for completion around a specific outcome

They measure outcomes and are used to show progress toward goals

They specify who, will do what, under what condition, by what standard and within what time period

How to Set Outcome Objectives There’s no magic number

e.g. 80% or 90% What is reasonable? What can you afford? What realistically can your staff accomplish? What percent shows you’re not committed and

what percent shows you’re naïve?

Why is This Hard? Because it is education

Because the best results may not happen for years

Because we are so busy doing what we are doing…. we forget why we are doing it

Challenges Identifying and defining outcomes is the easy

part. The devil is in the details. How do we track it, where does it all go, how

do we score it, compile it, turn it into a comprehensive report.

How do we “demonstrate improvement in institutional quality.”

Things to Remember

Outcome measurement must be initiated from the unit/department level (promotes ownership of process).

Measure only what you are teaching or facilitating.

Measure what is “important” to you or your program.

Be selective (2-3 outcomes only for a course, a select list for programs and institutional outcomes).

Put as much time in to “thinking through” the tracking process as you do into the definition of outcomes.

Spend the time up front in planning and the process will flow smoothly.

It will prove to be energy well spent.

Five Dimensions of Good Assessment

Source: Linda Suskie, Middle States Commission on Higher Education

Good Outcome Assessment is Used The results are used to inform important

decisions on important goals. What is the department most concerned with? Are there goals for which the department/unit

needs data (proof)? Don’t create data and a document that sits on the

shelf – it should be used again and again.

Used, cont. Good assessment is planned and purposeful.

Who has been/will be involved in the decision about what to assess?

When will assessment occur (best time, best course)?

What will you do with result or what results do you want to see?

What should be measured first? If you add more variables each year, what should come next?

Used, cont. Good assessment should focus on clear and

important goals Stay away from the vague It is clear that English should measure writing

skills It is clear that Speech should measure oral

communication What is it clear that you should measure?

Used, cont. Active participation of stakeholders

Get input from the right people State agencies, accrediting agencies, other colleges

with programs like yours Your faculty, staff and students Advisory committees and employers of graduates Anyone else?

Used, cont. Assess teaching-learning process as well as

outcomes What is the process and why is it important?

Methods, classroom strategies, online and hybrid –

Be concerned with more than the content – through which processes do student learn best?

Used, cont. Results communicated widely and

transparently What do you do if they are not learning? If your results are not good? Is that bad? Who needs to know the results? What does it mean to be transparent?

Used, cont. Results used fairly, ethically and responsibly

Faculty should not penalized Outcome data makes people territorial What does it mean to use outcome results

ethically and responsibly

Good Assessment is Cost Effective Cost effective – efficient and economically

(especially time) Don’t kill your faculty What are they hired to do? Do as little assessment as possible and still obtain

meaningful, good data What can you afford (both time and money)?

Cost Effective, cont. Focus on clear and important goals

Pick 3-6 goals at the maximum for which to create outcome assessment

All the courses in your department/program contain hundreds of objectives.

Which ones are most important?

Cost Effective, cont. Start with what you have

What do you have? Assignments and tests?

Can you work with those? (examples)

Group projects Clinical hours Co-op and internship experiences

Cost Effective, cont. Simple

Don’t create an elaborate assessment process unless you have enough staff and resources to carry it out

If you create surveys or assessment tools, keep them simple

If you conduct focus groups, get as much out of them as possible with as few questions as possible

Cost Effective, cont. Realistic expectations

What should your benchmarks be? Will it all “go down” like you want it to? What will happen if faculty have trouble with the

process? Should you start with training?

Good Assessment is Accurate and Truthful Flow from clear and accurate goals

Gives us an accurate perception of what is really happening

Accurate and Truthful, cont. Represent a balanced sample of key goals

including thinking skills Measures what is important Measures what are considered “critical” to

students in the field Measures more than what students can memorize

– but rather their processing and thinking skills

Accurate and Truthful, cont. Use a variety of approaches, including direct

evidence of student learning Tests and classroom assessments Surveys of graduates and employers Focus groups Portfolio assessments Direct observations and skill-based assessments

Accurate and Truthful, cont. Recognize diverse approaches to teaching and

learning and is developed thoughtfully Faculty who lecture, use small groups, interactive

approaches – will it work for all? Take the time to make sure this assessment will

give you the results you wanted.

Accurate and Truthful, cont. Perpetual works in progress

No one does this well the first time Health programs have been doing it the longest

and can help others It takes more programs a couple of times before

they create good assessment data Do you have that kind of time? Our attitude should be “we are always growing

and improving.”

Good Assessment is Valued Assessment results/efforts are

recognized/honored Who should value the results If they are valued, what should our actions be What about senior administration (part of a

process) If we uncover needs, what should happen?

Valued, cont. Innovation, risk-taking and efforts to improve

teaching and learning are recognized and honored If faculty must do assessment and are willing to

try new things to improve results, we should honor and value their efforts.

Assessment will have direct effect of a center for teaching and learning.

Valued, cont. Supported with appropriate resources: time,

guidance, support and feedback What do we need to do this well? What should happen to the results?

Clear and Important Goals Have clear and important standards for

acceptable and exemplary student performance Do we test for minimal standards? What is considered good performance? Are there clear standards? Should we compare ourselves with other schools?

An Example – English 111 Major Competencies

Develop competency in writing brief essays of three types: descriptive, narrative and analytical

Develop competency in writing Standard English also Practicing Grammar and Usage

Explore multiculturalism through analytical reading.

ENG 111 and Good Assessment Good Outcome Assessment is Used Good Outcome Assessment is Cost Effective Good Outcome Assessment is Accurate and

Truthful Good Outcome Assessment is Valued Good Outcome Assessment Comes from

Clear and Important Goals

But Let’s Make Sure We Cover Our Bases with SACS:

What Do We Need to Do?

SACS Requirements Core Requirement 2.5 The institution engages in ongoing, integrated, and

institution-wide research-based planning and evaluation processes that 1) incorporate a systematic review of institutional mission, goals and outcomes; 2) results in continuing improvement in institutional quality and 3) demonstrates that the institution is effectively accomplishing its mission (Institutional Effectiveness).

Comprehensive Standard 3.3.1 The institution identifies expected outcomes, assesses

whether it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas:

3.3.1.1 educational programs, to include student learning outcomes

3.3.1.2 administrative support services 3.3.1.3 educational support services 3.3.1.4 research within its educational mission, if

appropriate 3.3.1.5 community/public service within its

educational mission, if appropriate

Comprehensive Standard 3.5.1 The institution identifies college-level general

education competencies and demonstrates that graduates have attained them.

What We Need to Have in Place An institution-wide research-based planning

and evaluation processes A systematic review of institutional mission,

goals and outcomes Results in continuing improvement in

institutional quality The ability to demonstrate that the institution

is effectively accomplishing its mission

What We Need to Have in Place. Identifies outcomes in educational programs

(program and individual level), administrative support areas and educational support services.

Assesses whether we achieve these outcomes, and provides evidence of improvement based on analysis of the results in each areas.

Established general education competencies and demonstrates that graduates have attained them.

Your Program Review Each program will:

Establish learning and program outcomes Assess those outcomes Identify benchmarks or objectives for those

outcomes Identify strengths and weaknesses Make recommendations for changes Use the results to make improvements (strategies

for change)

Your Have Institutional learning outcomes

Will need outcomes for the rest of the college Have a program review model (evaluation process)

Need to make sure ESS and Admin Services are on board After this workshop, you will have program and

learning outcomes down to the program level. Make sure Gen Ed assessments are included

Let’s Begin