1/17/2012
1
Developing and Writing Program-Level Student Learning Outcomes andMapping those Outcomes in the Curriculum
Jeremy Penn, Ph.D.Director, University Assessment and Testing
University Assessment and Testing
Faculty Certificate Program / Graduate Student Endorsement in Program Outcomes
Assessment
• Participate in 8 out of 10 workshops– 9 offered in 2011-2012
– Some limited substitutions allowed
• Create or modify an assessment plan for a program (can be hypothetical)
• Certificate and $500 (faculty) $100 (graduate student) award upon completion
University Assessment and Testing
Identify student learning outcomes
Identify or design tasks to assess
learning
Analyze results
Use results for improvement
Assess the assessment – revise
and reengage
1/17/2012
2
University Assessment and Testing
University Assessment and Testing
Identify Student Learning Outcomes
• Perhaps the most important element of the process
• Without clear, well-written outcomes the remaining steps lack focus and direction
• Developing these outcomes can be a valuable improvement process in and of itself
University Assessment and Testing
Sources of Learning Outcomes
• Can be developed from:– Mission statements
– Accreditors / professional organizations
– Student work
– Taxonomies
– Faculty members’ expectations for student achievement
– Other sources
1/17/2012
3
University Assessment and Testing
Starting the Conversation
• Who should be involved?
• Why is it important to have this conversation?
• What do I want to come out of this conversation?
• When should we have this discussion?
University Assessment and Testing
Involving Faculty
• Consider what “involvement” really means
• How are faculty members involved in establishing the curriculum?
• Reflect upon different levels and types of involvement – not everyone must be involved in everything
• A single person shouldn’t be doing it all
University Assessment and Testing
Discussion Starters
Cognitive
AffectivePsychomotor
• What do we expect students to learn in these three domains?
• What would an ideal graduate from our program look like?
• In what area(s) of student learning do we feel there are concerns?
• What skills, abilities, knowledge, dispositions should students have when they graduate?
Diagram from Maki (2004)
1/17/2012
4
University Assessment and Testing
Outcome Types
• Input outcomes (# applications per semester)• Student needs outcomes (identify student needs)• Service utilization outcomes (use of facilities /
services)• Student satisfaction outcomes (satisfaction with
services provided)• Student learning outcomes
– Depict skills, knowledge, abilities, and affective dimensions that students are expected to achieve
University Assessment and Testing
Why Student Learning Outcomes?
• Other outcome types may also be important
• Student learning and development is a key element of educational programs
• Accreditors expect evidence of student learning (in addition to other types of data)
• Learning matters most!
University Assessment and TestingFrom Maki (2004)
1/17/2012
5
University Assessment and Testing
Examples
• Students will be able to develop and express ideas through writing in a variety of genres and styles.
• Students will be able to analyze the meaning of major texts from both Western and non-Western cultures.
• Students will be able to perform music from a variety of historical / style periods and exhibit the appropriate skills for musical self-expression.
• Students can recognize common biotic and abioticstresses and options for reduction of stresses with minimal disturbance.
More examples on uat.okstate.edu
University Assessment and Testing
Writing Student Learning Outcomes
• Some authors suggest using a template: – “Students will be able to
[action verb] [object].”– Can be overly
prescriptive– Assessment as “mad
libs”
• Instead, focus on key elements of high-quality outcomes
University Assessment and Testing
Elements of High Quality Outcomes
• Meaningful• Manageable• Focuses on what students will know or be
able to do (not what faculty or staff will do)• Uses action verbs• Connects with the mission of the institution /
program / course• Will provide evidence that will lead to a decision for
continuous improvement
1/17/2012
6
University Assessment and Testing
Balancing Breadth, Measureable-ness, and Meaningful-ness
Compre‐hensive
Meaning‐ful
Goal
Simple
But too broad to be meaningful or assessable
But not meaningful nor important But difficult to assess
University Assessment and Testing
Activity: Improving Student Learning Outcomes
1. Students will be satisfied with their opportunities to learn critical thinking.
2. Faculty members will teach students key elements of writing a research paper.
3. Students will become life-long learners. 4. The number of students participating in
multicultural activities will increase by 2%.5. The number of books in the library will reach
500,000 by 2011.
University Assessment and Testing
Activity
Practice developing program-level student learning outcomes
1/17/2012
7
Part 2
Curriculum Mapping
University Assessment and Testing
What educational experiences are we providing to students to help them achieve these outcomes?
University Assessment and Testing
Is it fair…
…to expect students to achieve some outcome (like learning to write)…
…if students don’t have the opportunity to learn that outcome and receive feedback on their progress?
1/17/2012
8
University Assessment and Testing
Scenario
Writing listed as key learning
• Student writing is listed as a key learning outcome– No one in the degree program is teaching
writing– Two possible causes:
• Perhaps writing isn’t that important!• Perhaps the department needs to make a
curriculum change to include opportunities to learn how to write
University Assessment and Testing
Types of Experiences
• Coursework• Practica• Internships• Co-curricular activities• Clubs, campus speakers, events• Use of library, labs, technology, projects• Undergraduate research experience• Others
University Assessment and Testing
It can be very revealing to simply write out the educational experiences that are offered to students to help them achieve the student learning outcome!
1/17/2012
9
University Assessment and Testing
Having a description about the educational experiences:– shows whether or not it is reasonable to
expect students to achieve the outcome– reveals areas to improve if there are
achievement issues identified– is an important part of program self-reflection– assists in identifying where information about
student achievement of the student learning outcomes can be gathered (future workshop)
University Assessment and Testing
Principles in Organization
• Continuity– Vertical reiteration (multiple opportunities)
• Sequence– Successive experiences build on the preceding
one and go more broadly and deeply
• Integration– Horizontal relationship to support a unified view:
not isolated behaviors
University Assessment and Testing
To what extent are continuity, sequence, and integration evident in your degree program?
1/17/2012
10
University Assessment and Testing
I = IntroducedR = ReinforcedE = Emphasized
Course orEducational Experience
Course or Educational Experience
Course or Educational Experience
1. Student Learning Outcome One I I R
2. Student LearningOutcome Two I R
3. Student Learning Outcome Three E
From Maki (2004)
University Assessment and Testing
Next level: examine more carefully how (not just where) the learning outcomes are addressed
University Assessment and Testing
Activity
• Use the student learning outcome we developed earlier (or another student learning outcome of interest)
• Sketch out a basic curriculum map
1/17/2012
11
University Assessment and Testing
• Student learning outcomes– An art, not a science
– Meaningful, manageable, and reasonable
– Clearly communicate expected student learning (but don’t have to be comprehensive)
• Curriculum mapping– Not limited to coursework
– Can be very revealing
– Important element of planning to use results
– Not just where, but how
University Assessment and Testing
Upcoming Workshops
February 7th: Effective Alternative Assessment Practices (Presented by Pam Brown)
March 29th: Best Practices in Program Outcomes Assessment: Presentations by OSU Assessment Coordinators
April 10th: Analyzing and Interpreting Assessment Data (Presented by John Hathcoat)