Program Review, Service Unit Outcomes and Planning

Eva Bagg, Associate Dean of Institutional Effectiveness

Maria Narvaez, Planning Analyst

July 8, 2009

Agenda

• Overview• LBCC program planning & review• Timeline • Components of program planning & review• How to begin developing mission statement,

goals & SUOs• Purpose & principles of assessment• Unit assignment• Sneak preview of TracDat• Questions/Sign-up

Overview

• Today’s workshop will– Provide context for SUOs– Guidance to prepare content for program

planning and review• Customized Training sessions with each

administrative unit will follow today’s workshop– Sign up for a time that works for your unit

LBCC program planning & review:

a historical perspective

ProgramReview – every 3 yrs

Dept. Planning -No formal

process

SUOs

Non-Instructional

Dept. Planning -process

had problems

ProgramReview – every 6 yrs

SLOs, But no

assessment plans

Instructional

The new LBCC program planning & review process

Program Review Planning

Outcomes Assessment

Student Success &

Institutional Effectiveness

Timeline for first implementation of new

process• July – customized training sessions &

units draft SUOs• July/ August – feedback to units• August – hands-on support to enter

information into TracDat• Program Reviews are DUE on Oct. 1st *

• Program Plans/ Reviews submitted to Validation Teams and then to “inter-level” groups then to VP Level.

* We hope most Admin Units will have their program reviews completed by College Day.

Program planning & review

Mission Statement

• A unit mission statement should:– identify the unit’s purpose– reflect how the unit supports a student learning

environment – be distinctive– articulate the essential functions/activities of

the unit– identify the primary stakeholders of the unit– align with LBCC’s mission

• See handout for examples

Goals• Goals are:

– overarching general statements that describe the department's strategic direction.

– usually not measurable but rather, they represent coordinated efforts of intended improvements for the department’s processes

– what the department plans to do and are reflective of long-term priorities

– representative of consensus of staff aspirations for the program

– primarily used for general planning and are used as the starting point to the development and refinement of outcomes.

• See handout for examples

How to get started

(1) Meet with your unit to discuss and gain consensus on your unit mission– If your unit comprises multiple functional areas,

make sure each area is represented when you meet.

– Discuss the purpose of your unit (unit mission)• What are the primary functions or activities of your

unit?• Who are your stakeholders? Who do you provide

services to and/or who benefits from the services?

How to get started

(2) Meet with your unit to discuss and gain consensus on your unit goals– Review the goals stated in your last program

review.• How far have you come to achieving these goals? • Are they still relevant?

– Decide which previous goals your unit will retain (and/or refine).

– Write any new goals that state what you want to improve or modify about your unit in the next 3-5 years. (try to limit to 5)

What are Service Unit Outcomes?

LBCC’s definition of a SUO: Outcomes describe the desired quality

(timeliness, accuracy, responsiveness, etc.) of key functions and services within the support or administrative unit.

SUOs are expressed as statements of what users of the service experience, receive or understand as a result of a given service interaction. These should address both process and satisfaction indicators.

What are Service Unit Outcomes?

Types of SUOs

• Process outcomes– The level or volume of unit activity – The efficiency with which the unit’s processes are conducted – Compliance with external standards of “good practice in the field” or

regulations– Effectiveness of the service in accomplishing intended outcome

• Client learning can also be an outcome, but the results are used to improve the effectiveness of the unit

• Satisfaction outcomes– Satisfaction-oriented administrative objects range from overall

“client/customer satisfaction” as included on institutional graduating student surveys to more specific surveys of client satisfaction with components of individual services.

How to develop SUOs(1) Consider how your unit will determine the

extent to which it accomplishes its goals (What will you assess?)– Aim to develop SUOs that make your unit goals measurable– Use existing data as much as possible– Focus on essentials you can address– Incorporate assessment procedures into routine unit operations– Use a cycle of assessment so that all outcomes will be assessed

within a 3-year period rather than assessing all outcomes each year.

– Try to limit to 2 SUOs per functional unit

How to develop SUOs

(2) Understand the relationship between goals and outcomes

– Goals are what we plan to do. They are helpful for guiding the implementation of improvement activities but are sometimes too general, broad, or vague for developing specific tools to assess the effectiveness of services provided in support of student learning.

Goals state what we will do or cause to happen.– Outcomes are what we expect will happen when we do what we

planned. At a minimum, outcome statements should describe stakeholder behaviors that staff would accept as evidence that the support office is performing its function. Where possible and meaningful, write SUOs beginning with the client as the noun in the statement (ie., the client will receive or experience…).

Outcomes state what will happen as a result of what we do.

How to develop SUOs

Goals Outcomes

Broad, future-oriented statements Brief statements of results

What the program or office is striving toward

Describe behaviors of stakeholders

Refer to process Refer to results of processes

Used in policy making and program planning

Used to assess effectiveness and provide improvement information.

Understand the relationship between goals and SUOs, continued

Practice exercise: Goals or outcomes

• Faculty, staff, and students will have access to electronic resources from off-campus locations.

• Learning Resource Services will increase access to its resources by implementing remote user access to electronic resources.

Practice exercise: Goals or outcomes

• The Office of Institutional Effectiveness will provide planning, assessment, and evaluation services to meet the needs of the College.

• Faculty, administrators, and staff use information to make decisions or assess the effectiveness of their area.

Practice exercise: Goals or outcomes

• The Office of Risk Services will promote peace, order, and safety on campus by deterring and preventing criminal activity.

• The number of crimes reported on both the LAC and PCC campuses will decrease annually.

• Faculty, staff, and students will feel safe when they are on the campus.

Writing an Outcome Statement:The 3 M’s

1. Meaningful: How does the outcome support the departmental mission or goal?

2. Manageable: What is needed to foster the achievement of the outcome? Is the outcome realistic?

3. Measurable: How will you know if the outcome is achieved? What will be the assessment method?

- StudentVoice

See handout for more SUO examples

Why assess?

The ultimate goal of assessment is to pose questions and gather information to

bring about improvement.

Why Assess?

Student Support and Administrative units• define their mission, • establish goals • identify service unit outcomes and develop plans to assess them, and

• collect assessment dataso that key processes that meet the needs and expectations of students, parents, employers, faculty and other stakeholders can be improved on a continuous basis.

Principles of assessment

Assessment at LBCC is• To enhance quality of services provided in support of teaching and learning

• Faculty and staff driven• Conducted at the level of the unit (not about individual evaluation)

• On-going and cyclical• Done in order to improve

Assessment plan

LBCC Program/Course/Unit Level Outcome and Assessment Plan Rubric

Program/Course/Area Name: Program Mission Statement:

Intended Outcome Assessment Task Criteria /Expected Level of

Achievement Results of Assessment Use of Data/Plan

(Continue grid for each program/course/unit outcome)

Assessment plan

LBCC Program/Course/Unit Level Outcome and Assessment Plan Rubric

Program/Course/Area Name: Risk Services Program Mission Statement: …….

Intended Outcome Assessment Task Criteria /Expected Level of

Achievement Results of Assessment Use of Data/Plan

1. The number of incidents of crimes reported on campus will decline each academic year.

1. The department of Risk Services will require that the Long Beach Police Department provide an annual report on crime statistics.

Each year the number of crimes reported overall at both campuses will decline by 3%.

2. Faculty, staff and students will report feeling safe when they are on campus.

3. Items on the LBCC employee survey will query faculty and staff on the extent to which they feel safe on campus.

85% of faculty and staff responding to the LBCC employee survey will report a level of “agreement” or higher that they feel safe on campus.

(Continue grid for each program/course/unit outcome)

Note: For each functional area within a unit, do only the first 3 columns for fall 2009.

Assessment planIntended Outcome Assessment Task

Criteria /Expected Level of Achievement

Results of Assessment Use of Data/Plan

Write the intended outcome in this column.Student Learning Outcomes (SLOs) are identified in the course outline of record. Program Level Outcomes, either SLOs or service unit outcomes (SUOs) are identified statements for a program/area that will be assessed to measure the effectiveness of the program/unit.

Explain the assessment task for each outcome in this column. Make sure that a balance of direct and indirect assessment methods are used program-wide. Describe what the assessment is, how the assessment task will be implemented, who will be responsible for the task, when the assessment will be conducted, and where activities will be carried out.

The expectations for success should be explained in this column. Establish a minimum expected score for achieving the outcome (e.g. %, fraction, actual number); determine the acceptable success standard (e.g. satisfactory, a passing grade, narrative indicator); as well as the students/clients to be included (e.g. successful completers, all queried, random sample). This information initially will be based upon reflection and discussion among necessary parties. Later, consideration of previous outcomes assessment effort(s), external reports (if available) should be used for this determination.Internal and external conditions should be considered.(Conduct and document assessment activities.)

Once the assessment data is collected analyze the information through small group discussion and in relationship to previous assessment data (if available). Report the actual results and compare with the original expectations in this column. Highlight key findings from the data. Develop supportable conclusions from this information. Determinations will then be made on what action should be taken.

Report here what the department/program/unit members have done or are doing as a result of the findings. Identify any changes or steps, who has made or is making the modifications and indicate the time frame for any changes to improve the outcome. The actions should generally be substantive and detailed; although there are times when no action is necessary and this can be stated as well. Indicate what the data explains about the process and what resources are necessary to implement the needed program or course improvements to student learning.(Conduct follow-up assessment to determine the effectiveness of the program or course improvement.)

Administrative Units Completing Program Plan/

Program ReviewSuperintendent-President's

OfficeInstitutional EffectivenessCommunity Relations and

Marketing

Academic Affairs Academic Services

Administrative ServicesRisk ServicesFacilitiesFiscal ServicesInstructional and Information

Technology Services

Human ResourcesClassified AcademicProfessional Development

Student Services LBCC AuxiliaryFinancial Aid/Veteran AffairsAdmissions & RecordsCounseling/ Student Support

Development Student Affairs Outreach and Recruitment

Economic & Resource Development

Economic DevelopmentWorkforce DevelopmentInstitutional Grants

Your Unit Assignment

• Pull together a work group for your unit– Composition: Ideally, everyone (directors,

manager, supervisors, classified staff) should be involved; however, if this is not feasible, ensure that there is representation for:

• each functional area • classified staff • if applicable, for each campus and/ or for each shift

Work group assignment

• Answer two questions about your unit:– What do you do? (core functions & services)

• You may want to start by having each functional rep describe their core function. From a comprehensive list of all functions and services, distill the core functions and services of the department.

– What do you hope to improve or modify in the next 3-5 years?

• Review goals from the last Program Review. Are they still relevant?

• Draft mission statement• Draft goals for next 3-yr cycle.

Work Group Assignment

For each functional area within your unit• Draft SUOs

– Start by jotting down possible results of your goals (make your goals measurable)

• (Optional) For each of your SUOs, complete the “Assessment Task” and “Criteria for Success” columns of the Assessment Plan template

End Products from Work Group

• Draft department mission statement• Draft department goals• Draft SUOs (for each functional area

within your department)

Please have these ready and available for the customized training session.

Note: We will send you a template in which to capture the above end products from your work group.

Customized Training

• Feedback and refinement of mission, goals and SUOs

• Assessment plan• Introduction toTracDat

– Identify point person in unit to enter info into TracDat

Program/Unit description screen

Goals screen

Outcomes screen

Questions?

• Sign up for customized training