ASSESSMENT PEER CONSULTANT TRAINING St. Cloud State University January 28 and 29, 2008 Trainers:...

ASSESSMENT PEER CONSULTANT TRAINING

St. Cloud State University

January 28 and 29, 2008

Trainers: Patricia Aceves, Elaine Ackerman, Wendy Bjorklund, Lisa Foss, Sandra Johnson, Jim Knutson-Kolodzne, Joe Melcher, Jim Sherohman

www.stcloudstate.edu/assessment

http://www.stcloudstate.edu/assessment

INTRODUCTIONS OF PARTICIPANTS AND TRAINERS

Introductions

Name and current work assignment Why you want to be an assessment

peer consultant A concern, question, or apprehension

you have about being an assessment peer consultant

Your impressions of Walvoord’s assessment workshop

INTRODUCTION TO PEER CONSULTANT TRAINING

HLC Assessment Academy

4-year commitment Exempts institutions from

accreditation follow-up visits on assessment

Centers on completion of student learning projects

SCSU’s Assessment Academy Projects

University Assessment System Assessment of Student Learning in

Programs General Education Assessment

www.stcloudstate.edu/assessment/academy.asp

http://www.stcloudstate.edu/assessment/academy.asp

http://www.stcloudstate.edu/assessment/academy.asp

Goals of University Assessment System Project

1) Refine institutional assessment policies2) Communicate institutional assessment

policies 3) Build institutional assessment capacity4) Integrate assessment into work life of

institution5) Implement assessment of assessment

Goals of Project on Assessment of Student Learning

in Programs

1) Implement institutional assessment plan at level of major programs

2) Implement institutional assessment reporting system at program level

“Train the Trainers” Workshop, May 2007

Did initial planning for peer consultant training

Facilitated by Academy mentor Elaine Klein

20 participants from across the campus

Goals of General Education Assessment Project

1) Approve the program2) Implement program structure3) Collect and analyze data4) Use data for improvement

Goals of the Peer Consulting Program

Improve program assessment across the institution

Build assessment capacity Increase acceptance of assessment of

student learning at the program level

Learning Outcomes for Peer Consultant Training

Those who complete this training will be able to:

Articulate the basic principles of assessment

Present the SCSU model of assessment Facilitate the assessment process in

departments, programs, and units

CONSULTANT EXPECTATIONS AND THE ASSESSMENT PEER CONSULTING PROGRAM

Assessment Peer Consultants

Assist with any aspect of program-level assessment

Respond to requests from programs Work with at least one other peer

consultant Are familiar with the resources available

at SCSU and use these when appropriate

Represent the SCSU Assessment Peer Consulting Program and the Assessment Steering Committee

Peer Consulting Process

Request submitted to Assessment Office Assessment Director contacts program to obtain

additional information about the request Assessment Office selects two (or more) peer

consultants with complementary backgrounds Peer consultants meet, then contact the

program Peer consultants assist the program Peer consultants provide brief descriptive report

to the Assessment Office Program evaluates the peer consulting service

Peer Consulting Requests

Peer Consulting Request form How will peer consultants be

assigned? Can peer consultants decline

assignments? What are some good reasons for doing so?

Some Tips for Peer Consultants

1) Learn about the program before you visit.

2) Focus on the problem as defined by program faculty/staff.

3) Early on, ask questions that will help you understand the situation from their perspective.

4) If you are not actively facilitating, observe and take notes.

5) Help programs find a way to do assessment work themselves; don’t do it for them.

6) Nudge the program in the direction of recommended policies, to the extent this is appropriate.

7) Request help when needed.

Peer Consultants as a Team

Should there be a “lead” consultant? How will you share responsibility—drawing

upon each other’s strengths, while allowing opportunities for each to learn?

How will you communicate in the presence of program faculty/staff?• Support each other• Avoid the appearance of “ganging up” on

program faculty/staff• Comment on partner’s line of action without

offending program faculty/staff or the partner

CONSULTANT TRAINING:

ASSESSMENT PRINCIPLES

Definition

Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for

learning quality; systematically gathering, analyzing, and

interpreting evidence to determine how well performance matches those expectations and standards;

using the resulting information to document, explain, and improve performance.

Assessment

Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for

learning quality; systematically gathering, analyzing, and

interpreting evidence to determine how well performance matches those expectations and standards;

using the resulting information to document, explain, and improve performance.

Walvoord’s Definition

the systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning

(Barbara Walvoord, Assessment Clear and Simple)

Assessment’s core questions

What do you want students to know/do/value as a result of completing your program?

How do you know whether your students know/do/value these things?

If they know/do/value these things…you’ve got something to tell!

If NOT, you have useful information to guide changes to improve learning.

Where do programs begin in developing an Assessment Plan?

Consultants may assist:

Agree on Mission Identify Program Goals

Department GoalsProgram Educational Objectives Standards and Elements

ASSESS

Student Learning Outcomes

Data Collection and Interpretation Areas and

Methods of Assessment

Changes to Enhance Student Learning

MnSCU Work PlanMnSCU, SCSU, College Mission and Goals Professional/Accrediting Criteria Constituencies Recommendations Department Mission

Strategies to Accomplish Outcomes

ASSESS


Student LearningOutcomes


Areas and Methods of Assessment

Data Collection and Interpretation

ASSESSMENT – THE BOTTOM LOOP

STUDENT

Walvoord’s Three Steps of Assessment

1. Articulate your goals for student learning

“When they complete our program, students will be able to . . .”


What should consultants know?

At SCSU we say:

Identify your student learning outcomes

Students will <<action verb>> <<something>>

Student Learning Outcomes

• Learner Centered• Specific• Action oriented• Cognitively appropriate at the program

level

Possible Outcomes

• Students will appreciate….• Students will be exposed to….• Students will demonstrate knowledge of

the history, theories and applications of…

• Students will attend….• Faculty will provide students with

opportunities to….

Possible Outcomes

• Students will appreciate…• Students will be exposed to….• Students will demonstrate knowledge of

the history, theories and applications of…• Students will attend….• Faculty will provide students with

opportunities to….

KNOWLEDGECOMPREHENSION

APPLICATION ANALYSIS SYNTHESISEVALUATION

CiteCountDefineDraw

IdentifyList

NamePointQuoteReadReciteRecordRepeatSelectState

TabulateTell

TraceUnderline

AssociateClassify

CompareComputeContrast

DifferentiateDiscuss

DistinguishEstimateExplainExpress

ExtrapolateInterpolate

LocatePredictReportRestateReview

TellTranslate

ApplyCalculateClassify

DemonstrateDetermineDramatize

EmployExamineIllustrateInterpretLocate

OperateOrder

PracticeReport

RestructureScheduleSketchSolve

TranslateUse

Write

AnalyzeAppraiseCalculate

CategorizeClassify

CompareDebate

DiagramDifferentiateDistinguishExamine

ExperimentInspect

InventoryQuestionSeparate

SummarizeTest

ArrangeAssemble

CollectComposeConstruct

CreateDesign

FormulateIntegrateManageOrganize

PlanPrepare

PrescribeProduceProposeSpecify

SynthesizeWrite

AppraiseAssessChoose

CompareCriticize

DetermineEstimateEvaluate

GradeJudge

MeasureRankRate

RecommendReviseScoreSelect

StandardizeTest

Validate



CiteCountDefineDraw

IdentifyList


TabulateTell

TraceUnderline

AssociateClassify






TellTranslate




OperateOrder

PracticeReport


TranslateUse

Write


CategorizeClassify

CompareDebate


ExperimentInspect


SummarizeTest

ArrangeAssemble


CreateDesign


PlanPrepare


SynthesizeWrite


CompareCriticize


GradeJudge

MeasureRankRate


StandardizeTest

Validate

Lower division courseoutcomes



CiteCountDefineDraw

IdentifyList


TabulateTell

TraceUnderline

AssociateClassify






TellTranslate




OperateOrder

PracticeReport


TranslateUse

Write


CategorizeClassify

CompareDebate


ExperimentInspect


SummarizeTest

ArrangeAssemble


CreateDesign


PlanPrepare


SynthesizeWrite


CompareCriticize


GradeJudge

MeasureRankRate


StandardizeTest

Validate

Upper divisionCourse / Program

outcomes

Outcome: Reason Scientifically – Geology

What is asbestos? Explain how the characteristics of

amphibole asbestos make it more conducive to producing lung damage than other fibrous minerals.

Given the formula Mg3Si2O5(OH)4, calculate the weight percent of magnesium in chrysotile.

Outcome: Reason Scientifically – Geology

Two controversies surround the asbestos hazard: (1) it is nothing more than a very costly fabrication all out of proportion to actual risk, or (2) it is a serious hazard that accounts for tens of thousands of deaths annually. What is the basis for each argument?

What is your role as a consultant?

ASSESS





STUDENT

Strategies to accomplish

Identify the assignments and activities that demonstrate

achievement of each learning outcome

Consulting at this stage?

StudentLearning Outcomes Course

1

x

x

Course2

x

x

x

x

Course3

x

x

Course4

x

x

x

x

Other

x

x

x

Program Matrix


2. Gather evidence about how well students are meeting the goals.

Direct measures Indirect measures

“Evidence includes qualitative as well as quantitative information.”


Gather Evidence Using Direct Measures

Directly evaluate student work“Exams, papers, projects, computer

programs,interaction with a client, musical

performance”


Gather Evidence Using Indirect Measures

Asking students or alumni how well they thought they learned

Tracking graduate school or job placement rates


Challenges in Gathering Evidence

Recognize appropriate measures to use

Data from multiple courses and instructors - interpret, report, and utilize in decision making

Relate to outcomes when course or instructor-specific assignments and activities are used

Possible Consulting Requests

Identify the traits of successful achievement of

the outcome What are we looking for to be able to tell

whether or not student performance is acceptable?

Example

OUTCOME: Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected

BETTER: Students will apply factual information to a problem.

TRAITS: Relevance Clarity

Comprehensiveness Awareness of Bias

Possible Consulting Requests

Identify characteristics ofeffective/accurate/successful

performance for each trait

• Scale or description for assessing each of the traits

• Two to five-point scale for each trait

Performance Rubric

ExceedsExpectations

MeetsExpectations

Does not meetExpectationsTraits

Performance Characteristics

Descriptionsof

Unacceptable,Acceptable,or ExcellentPerformance

Performance RubricBusiness Management

ExceedsExpectations

MeetsExpectations

Punctuality

Team’s Customer Satisfaction Skills

Performance CharacteristicsDoes not meetExpectations

Courtesy

Communication

Some team members missed appointments or

did not return phone calls.

Some team members were not respectful of

firm employees

All team members were always courteous

and respectful of all firm employees.

All team members

were always early.

All team members arrived on time for

appointments and returned all phone

calls promptly.

All employees felt that The team members

were very courteous and respectful and fully

elicited their ideas.

Some team members did not communicate

clearly during meetingsand phone calls.

The team membersalways communicated

clearly during meetingsand phone calls.

The team members always made an extra

effort to make sure that they understood

us and that we understood them.

ASSESS





Data Collection and Interpretation


STUDENT


3. Use the information for improvement

Pitfall: Gathering data that no one will use “It is not enough to gather data about student

strengthsand weaknesses; you need information and

hypothesesabout the causes of student weaknesses.”


The Final Challenge

Close the loop Use data – develop hypotheses Review assessment methods Change curriculum and/or instruction

Possible consulting issues?

Thinking Outside the Loop

Accredited Programs Special Consideration?

Programs? Departments? Colleges?


More Pitfalls: Mere compliance with external

demands Triggering resistance and hostility of

faculty Letting administrators do it Making the process too complicated



It’s a passing fad It’s not my job or concern It violates academic freedom It violates student privacy You can’t assess what we do in academe I already assess - it’s called ‘grading’ I don’t have time to do assessment –

workload! I don’t have the expertise Assessment might reveal “bad” news

INSTITUTIONAL POLICIES AND RESOURCES

SCSU Assessment Structure

Organization chart Assessment Steering Committee Assessment Office Institutional Effectiveness Continuing Studies Student Life and Development Other areas

SCSU Recommended Policies

Assessment Plans Annual Assessment Reports

These policies apply to all undergraduate major programs, graduate programs, and free-standing minor programs

These policies have not been approved by the Faculty Senate, but they are recommended by the Assessment Steering Committee

Assessment Plan

Mission Student Learning Outcomes Program Matrix – shows which courses or

outside-of-course activities will be used to assess each student learning outcome

Timeline – identifies when (which year) these assessments will take place

Annual Assessment Report

Student learning outcomes assessed Methods and tools used to assess

these outcomes Summary of findings for these

outcomes Discussion of findings Use of findings for improvement Plan for next year

Other Institutional Policies

General education www.stcloudstate.edu/generaleducation/assessm

ent.asp

Upper division writing http://www.stcloudstate.edu/generaleducation/do

cpolicies/UpperDivisionWritingRequirement.asp

Institutional learning outcomes? http://www.stcloudstate.edu/actionplanning/acad

emic/workgroups/outcomes.asp

OFFICE OF INSTITUTIONAL EFFECTIVENESS (OIE)

Office of Institutional Effectiveness (OIE)

The Office of Institutional Effectiveness strives to facilitate the integration of planning, institutional research, and assessment to encourage the continuous and dynamic

improvement of the University’s programs and services.

The goals of the office include: Create a more balanced and transparent planning

process Encourage broad discussion and understanding of

university priorities Enhance accountability to internal and external

stakeholders Expand use of data in decision-making & budgeting Encourage and support evaluation and assessment

that leads to continuous organizational improvement

OIE areas of responsibility

Institutional Research: http://www.stcloudstate.edu/oie/research/default.asp

SCSU Strategic Planning: http://www.stcloudstate.edu/oie/planning/process/default.asp

SCSU Action Planning: www.stcloudstate.edu/actionplanning

Minnesota State Colleges & Universities reporting

Data & reports from OIE

Enrollment reports Retention and graduation 10th & 30th day enrollments Student profile

Common Data Set Fact Book and Department Data Reports Graduating Senior Survey Custom data requests

(http://www.stcloudstate.edu/oie/research/datarequest.asp)

Nationally normed reports from OIE

National Survey of Student Engagement Self-reported data on student experience of

freshmen and seniors Administered spring of odd years (‘01, ‘03, ‘05,

‘07) Collegiate Assessment of Academic Proficiency

(CAAP) Measures critical thinking and writing Administered spring 2007

Collegiate Learning Assessment (CLA) Measures critical thinking and writing Administered fall 2007 (freshmen); spring 2008

(seniors)

University Data System

The University Data System (UDS) is being created to enhance the use of data in university planning and budgeting, to streamline access to disparate data

sources for institutional decision makers, and to increase accountability to internal and external stakeholders. The

development of the UDS will contribute to evidence-based planning and resource allocation, and to a culture

of continuous improvement at SCSU. Cubes currently under development

Instructional Workload (May 2008) Assessment (November 2008) Enrollment Management & Retention (May 2009)

OIE staff & contact information

Lisa Helmin Foss, AVP for Institutional [email protected]; 320.308.4028

Director of Institutional Research (vacant)

Deborah Bechtold, Research [email protected]; 320.308.3039

David Sikes, Research [email protected]; 320.308.1746

Links to data and reports and information on the Office of Institutional

Effectiveness

www.stcloudstate.edu/oie

STUDENT LIFE AND DEVELOPMENT ASSESSMENT 2007

Student Life and Development

The Division of Student Life and

Development

has established learning outcomes and

translated them into understandable

outcomes for our students.

Student Life and Development Learning Outcomes

Diversity and Global Environment.

The world we live and interact in is diverse and

requires us to interact globally. To effectively

engage in today’s world students will learn to…


Understand uses of power and nature of oppression,

be aware of cultural and personal differences

Trust and respect others, empathize, access

culturally appropriate resources

Be committed to cross-cultural communication,

value social responsibility, be committed to social

justice


Inter and Intra-Personal Competence.

Developing awareness of ourselves and the way

we interact with others are essential

characteristics. The more we know about our

strengths and weaknesses and the impact of our

behavior on others, the better we are able to

develop healthy relationships. To facilitate this

growth students will learn to…


Gain knowledge about themselves, develop consciousness of

self, self-confidence, feelings of mattering, manage personal

emotions, value cultural heritage

Apply self-knowledge, practice self-worth , congruence,

commitment, identify passions, discuss cultural differences and

issues

Work with others, practice collaborations, controversy with

civility, engage across difference, be committed to ethical

action


Engagement/Social Responsibility and

Community Building.

it is essential that we learn what it means to work

together toward a common good. These skills are

important whether on campus now or in our future

communities. It is also imperative that we understand

our responsibility as educated citizens in our

community. Toward this end students will learn to…


Create effective change and practice collective efficacy

Develop common purpose

Develop civic awareness, value civic responsibility, practice engaged citizenship

Develop a sense of connectedness within one’s communities


Cognitive and Intellectual Growth.

The intellectual and practical skills we need are

extensive, sophisticated and expanding with the

explosion of new technologies and increasingly

complex societies. As we progress through the

educational system it is imperative to develop

competencies that synthesize specific skills into

higher level reasoning capabilities. To develop

these qualities students will learn to…


Engage with others in constructive ways, engage in

principled dissent; accept and appreciate other world

views; manage conflict constructively

Develop critical thinking/ decision making skills.

Be open to change

Foster lifelong learning.

Integrate academic knowledge into all aspects of living.


Leadership Exploration and Development.

In today’s world there are many situations in which

we will find ourselves in roles requiring leadership

skills. While these skills ultimately develop through

training, practice and mentoring, understanding

what it means to be a leader is multifaceted. To

gain these skills students will learn to…


Develop personal leadership skills, set individual

goals, practice risk-taking, delegate, serve as a

role model, manage people and tasks, facilitate

group processes

View leadership as a process, not a position


Critically evaluate leadership models, develop

group leadership skills, effectively lead change,

develop team building skills, develop leaderships

in peers, identify common purpose in groups, help

groups set goals, apply problem solving

strategies, value recognition and organizational

sustainability


Life Skills Competence. Life teaches us

many things, though learning to live with purpose

and responsibility toward self and others can be

quite challenging. Despite these challenges,

living a satisfied and competent life provides

fulfillment and direction for ourselves and those

with whom we come in contact. In order to gain

these skills students will learn to…


Develop effective communication skills and practice effective written and oral communication

Develop wellness skills

Develop life and career path

Understand and develop professionalism and self responsibility

Be globally-aware and support diversity.

Be self-aware and strive for meaningful relationships.

Be an engaged member of my various communities.

Take full advantage of opportunities for learning.

Explore leadership.

Live a balanced and purposeful life.

As an SCSU student I will:


Each department of the division is

currently in the process of establishing

departmental and programmatic learning

outcomes and identifying methods of

assessment.


The SLD assessment plan includes:

a process for reviewing department assessment plans

an Assessment Skills survey

an audit of assessment practices

department feedback process

training and professional development to help support the faculty and staff skill sets


Essentially, the challenge is to provide

support for the division to develop the

competencies inherent to a culture of

assessment.

The division must plan proper support

for competency development and

resources to inspire confidence and

move the division forward with regard to

assessment, measurement, and effective

methodologies.



It is our intent to secure resources to

jumpstart the required competency

development; identify needs and assess

our competency as a division; and

provide ‘in house’ support to help each

member of the division.


The division is eager to move forward, but

has a limited number of personnel who are at

a sufficient level of competency

The motivation and expectation is present,

but the need to measure who needs what

support and to provide competency

development is critical to an ‘all hands”

effort.


This; in turn, sets the expectation and

movement to make assessment

principles and practices a mainstream

part of all program entities in the SLD

division and also provides leadership in

leading SCSU in a integrated learning

direction for the benefit of the students.

Tools and Resources

ACPA ASK Standards Needs Assessment

Ten-Step Matrix in Student Affairs Assessment

Ten-Step Matrix in Student Outcomes Assessment

Assessment Practice in Student Affairs: An Applications Manual

Student Life & Development Assessment website

SWOT EXERCISE

SWOT Exercise

SWOT is a planning tool used to analyze the strengths, weaknesses, opportunities, and threats associated with meeting a particular strategic objective

Strengths and Weaknesses are internal to the organization or unit

Opportunities and Threats are external to the organization or unit

SWOT Discussion

Form groups of four. Your group should not include anyone else

from your department, program, or unit. Describe to the others in your group one

weakness of or threat to assessment in the program you analyzed.

Lead a discussion of how peer consultants might respond to this weakness or threat.

Allow an equal amount of time for each person (about 5 minutes each).

Group Reports

Which weaknesses and threats did you discuss in your small group?

Which of these were hardest to deal with? Why?

Do you see any interesting patterns that you would like to discuss?

Conclusion

Procedure for obtaining professional development funds from the Assessment Office

Biographical information for website? Evaluation of the training What do you feel most and least confident

about as an peer assessment consultant? Would you like additional professional

development on any topic related to assessment consulting?

Date post:	31-Mar-2015
Category:	Documents
Upload:	paxton-sabine
View:	216 times
Download:	0 times

ASSESSMENT PEER CONSULTANT TRAINING St. Cloud State University January 28 and 29, 2008 Trainers:...

Documents