Assessment of Assessment of Curriculum OutcomesCurriculum Outcomes

Dale WhittakerAssociate Dean and Director of Academic Programs

Purdue UniversityMay 21, 2008

The Purdue ExperienceThe Purdue Experience

Motivation:◦Program and Outcome Relevance◦Continuous Program Improvement◦Efficacy and Efficiency of Curriculum

Compliance: ◦University-wide Higher Learning Commission

Accreditation 2010◦Individual Program Accreditations (e.g. ABET,

SAF, IFT)◦5 year CSREES program reviews

Three-step ProcessThree-step Process

Identify Learning Outcomes and ActivitiesAssess Student Learning OutcomesImprove Student Learning

Step 1: Identify Learning Step 1: Identify Learning Outcomes and ActivitiesOutcomes and Activities

Outcomes◦What knowledge, skills and beliefs do we want

the students to attain as a result of this program?

◦Artifact: College and Program list of outcomesActivities

◦In what activities are students involved to help achieve this learning outcome?

◦Artifact: Matrix mapping activities to outcomes

College OutcomesCollege Outcomes

Program Outcome MatrixProgram Outcome Matrix

Matrix – How Program Activities Matrix – How Program Activities Address College OutcomesAddress College Outcomes

Step 2: Assess Student LearningStep 2: Assess Student Learning

Presentation by Christian Reiner

(Center for Instructional Excellence)

September 7, 2007

I taught Stripe how to whistle.

I don’t hear him whistling.

I said I taught him. I didn’t say he learned it.

It is assessment [that] helps us distinguish between

teaching and learning. (Leveque, 1999-2000)

DefinitionDefinition

“a set of processes designed to

improve, demonstrate, and

inquire about student

learning.”

(Mentkowski, M. qtd. in Palomba & Banta, 1999)

Key QuestionsKey Questions

What are you willing to accept as evidence/indicator that students have achieved the learning outcome?

To what degree have students achieved the outcome?

What facilitates/hinders students’ achievement of the learning outcome?

HOW to AssessHOW to Assess

Assessment Audit

Methods◦direct vs. indirect◦single vs. multiple (triangulation)◦standardized vs. self-developed

Assessment AuditAssessment Audit

Purpose: ◦Determine already existing assessment efforts

related to learning outcomes

Advantages:◦Saves time and other resources◦Validates what faculty do◦Shows that assessment is nothing new

Reporting ToolReporting ToolLearning Outcome

Course Measures Results Use of Information

COA 4: Communication

AG 299 Students write a final paper (5-10 pages) reviewing literature on a given topic. They hand in a 1st draft for feedback. The final paper is graded using a rubric and mastery is determined in 4 levels (beginning, developing, proficient, mastery). Criteria for the rubric are organization, mechanics, content knowledge, and analysis).

Students overall average performance is 2.14 (2.00 equals developing level). Their highest average performance was for content knowledge (2.44) and their lowest average was for analysis (1.87).

Based on the results, I focused more on helping students improve their analysis skills.

Adapted from Walvoord, 2004

References:

Walvoord, B. E. (2004). Assessment clear and simple: A practical guide for institutions, departments, and general education. San Francisco, CA: John Wiley & Sons, Inc.

Criteria

Audience

Accuracy

Rubric is taken/adapted from: ALTEC, the University of Kansas (2000-2007). Rubistar. Retrieved November 14, 2007 from http://rubistar.4teachers.org/index.php?screen=CustomizeTemplate&bank_rubric_id=127&section_id=5&.

Levels of AchievementCriteria Above

StandardsMeets

StandardsApproaches Standards

Below Standards

Audience

Accuracy

Rubric is taken/adapted from: ALTEC, the University of Kansas (2000-2007). Rubistar. Retrieved November 14, 2007 from http://rubistar.4teachers.org/index.php?screen=CustomizeTemplate&bank_rubric_id=127&section_id=5&.

Levels of AchievementCriteria Excellent Good Acceptable Failing

Audience

Accuracy

Checklist for Midterm Paper

Yes No

The paper is typed (not handwritten).

The manuscript is stapled.

The paper is at least 15 pages long.

The margins are 1 inch wide.

The text is left-aligned.

The text is double-spaced.

Rubric is taken/adapted from: ALTEC, the University of Kansas (2000-2007). Rubistar. Retrieved November 14, 2007 from http://rubistar.4teachers.org/index.php?screen=CustomizeTemplate&bank_rubric_id=127&section_id=5&.

Levels of Achievement

Criteria Above Standards

Meets Standards

Approaches Standards

Below Standards

Audience 7 points 5 points 3 points 1 point

Accuracy 4 points 3 points 2 points 1 point

Levels of Achievement

Criteria Above Standards Meets Standards Approaches Standards Below Standards

Audience 7 points 5 points 3 points 1 point

Demonstrates a clear understanding of the potential reader and uses appropriate vocabulary and arguments. Anticipates reader's questions and provides thorough answers appropriate for that audience.

Demonstrates a general understanding of the potential reader and uses vocabulary and arguments appropriate for that audience.

Demonstrates some understanding of the potential reader and uses arguments appropriate for that audience.

It is not clear who the author is writing for.

Accuracy 4 points 3 points 2 points 1 point

All supportive facts and statistics are reported accurately.

Almost all supportive facts and statistics are reported accurately.

Most supportive facts and statistics are reported accurately.

Most supportive facts and statistics were inaccurately reported.

Rubric is taken/adapted from: ALTEC, the University of Kansas (2000-2007). Rubistar. Retrieved November 14, 2007 from http://rubistar.4teachers.org/index.php?screen=CustomizeTemplate&bank_rubric_id=127&section_id=5&.

Levels of Achievement

Criteria Above Standards

Meets Standards

Approaches Standards

Below Standards

Audience 7 points 5 points 3 points 1 point

Demonstrates a clear understanding of the potential reader and uses appropriate vocabulary and arguments. Anticipates reader's questions and provides thorough answers appropriate for that audience.

Demonstrates a general understanding of the potential reader and uses vocabulary and arguments appropriate for that audience.

Demonstrates some understanding of the potential reader and uses arguments appropriate for that audience.

It is not clear who the author is writing for.

Accuracy 4 points 3 points 2 points 1 point

All supportive facts and statistics are reported accurately.

Almost all supportive facts and statistics are reported accurately.

Most supportive facts and statistics are reported accurately.

Most supportive facts and statistics were inaccurately reported.

Rubric is taken/adapted from: ALTEC, the University of Kansas (2000-2007). Rubistar. Retrieved November 14, 2007 from http://rubistar.4teachers.org/index.php?screen=CustomizeTemplate&bank_rubric_id=127&section_id=5&.

How Can Rubrics Facilitate How Can Rubrics Facilitate Learning? Learning?

Reveal standardsFoster self-assessmentEncourage ownership

◦Students help develop standards

◦Students help describe criteria

Invite communicationProvide different Insights

Huba, M. F., and Freed, J. E. (2000). Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Needham Heights, MA: Allyn & Bacon.

Tips for Using Rubrics Tips for Using Rubrics

Explain well Use for Assessment Refer to often Encourage Self-Assessment Facilitate reliable use

Huba, M. F., and Freed, J. E. (2000). Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Needham Heights, MA: Allyn & Bacon.

References:

ALTEC, the University of Kansas (2000-2007). Rubistar. Retrieved November 14, 2007 from http://rubistar.4teachers.org/index.php?screen=CustomizeTemplate&bank_rubric_id=127&section_id=5&.

Huba, M. F., and Freed, J. E. (2000). Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Needham Heights, MA: Allyn & Bacon.

Purdue ExperiencePurdue Experience

2007-08 – Assess Communications (Outcome 4)

Developed and adopted college-wide rubric

Reporting through Step 2

2008-09 Academic Year2008-09 Academic Year

2008-09 – Report on findings (Step 3) early fall

Expect course and curriculum modifications based on assessment evidence

Implement change and continue assessment

Assess College Outcome – Critical Thinking

University Wide Website – Spr 08University Wide Website – Spr 08

Thank you!dwhittak@purdue.edu

http://sharepoint.agriculture.purdue.edu/oap/outcomes