Student Learning Outcomes
Los Angeles Valley CollegeTraining, Spring 2008 – part II
SLO Coordinator – Rebecca [email protected]; (818) 947-2538
Review - Why SLOs?
New Accreditation Standards “Covering” material does not
guarantee students have learned it Success is determined by students
leaving a course/program with integrated, higher learning skills they can demonstrate
Establishes clear and transparent expectations for students
Review - What is this thing called SLO?
SLO means Student Learning Outcome.
They represent broad themes beyond specific course content.
They cut across the curriculum. They are measurable or observable.
Review - How is an SLO different from an objective?
Objectives Tied directly to
specific course content.
Address skills, tools, or content that enable a student to engage in a particular subject.
5 – 7 per course.
Outcomes Overarching
understanding and application beyond specific course content.
What students take away from the course that they can use in other courses or in life.
1 – 2 per course.
Assessment
What’s Assessment All About?
An ongoing process aimed at understanding and improving student learning.
Faculty making learning expectations explicit and public.
Faculty setting appropriate standards for learning quality.
What is Assessment All About?
Systematically gathering, analyzing and interpreting evidence to determine how well student performance matches agreed upon faculty expectations and standards.
Using results to document, explain and improve teaching and learning performance.
Tom AngeloAAHE Bulletin, November 1995
Roles of Assessment
“Assess to assist, assess to advance, assess to adjust” Assist – provide formulative feedback to
guide student performance Advance – summative assessment of
student readiness for what’s next Adjust – continuous improvement of
curriculum, pedagogy
Ruth Stiehl (2007)
Questions for Assessment
What do students need to DO “out there” that we are responsible for “in here”? (Stiehl)
How do students demonstrate the intended learning now?
What kinds of evidence must we collect and how do we collect it?
The Assessment Smorgasbord
When SLOs are well-written, the method of assessment is often clear.
One-size doesn’t fit all! To select appropriate tools, need to
understand: Types of tools available Nature of the data Potentials and limitations of each tool
Quality Data
Quality data: Based upon best practices Answer important questions Benefit the students & institution by
providing evidence to complete loop The assessment loop is a data-
driven method of decision-making. Questions are posed concerning
what works and what does not.
Quality Data: Are Results Valid and Reliable?
Valid - the data accurately represents what you are trying to measure. For instance the numbers of people that graduate don't necessarily represent good data on what has actually been learned.
Reliable - the data are reproducible. Repeated assessment yields the same data.
Authentic - the assessment simulates real-life circumstances.
Relevant - the data answers important questions, and is not generated simply because it is easy to measure.
Effective - the data contributes to improving teaching and learning.
Types of Assessment Data
Direct vs. Indirect
Direct What can the
student do or actually demonstrate they know
Can witness with own eyes
Setting is structured/ contained
Indirect What students say
they can do Things from which
learning is inferred Setting is not easily
structured/ contained
Qualitative vs. Quantitative
Qualitative Words Broad emergent
themes Holistic judgments Bulky to store and
report Often most valuable
and insightful
Quantitative Numbers Individual
components and scores
Easier calculations and comparisons
Easy to store and manage
Often has limited value
Must be carefully constructed to be valid
Formative vs. Summative
Formative Assessment for
learning “In-progress” Provide corrective
feedback Establish
foundational learning for next step
Summative Assessment for
evaluative purposes
“After the fact” Determine
progress/ achievement/ proficiency
Readiness for next step/role/learning experience
Criterion-based vs. Norm-Referenced
Criterion-based Evaluated/scored
using set of criteria Based on
proficiency not subjective measures such as improvement
Norm-referenced Assessment of
individual compared to other individuals or individual’s improvement over time
Rank, median Addresses overall
mastery but gives little detail about specific skills
Standardized vs. Homegrown
Standardized Assessments
created, tested, sold by an educational testing company
Usually scored normatively
Homegrown/Local Developed and
validated for a specific purpose, course, function
Usually criterion-referenced to promote validity
Embedded Assessments
Occurs within regular class or curricular activity
Class assignments linked to SLOs Individual questions on exams can
be embedded in numerous classes Immediate feedback
Grading vs. Assessing
Grading vs. Assessing
A course grade is based on student achievement of course objectives.
It is possible for a student to pass a class but not meet a specific course outcome and vice versa.
Various assessment techniques can be used in a class that may or may not be part of a grade.
Grading vs. Assessing
What would we look at to grade this assignment? (columns)
What would we look at for assessment? (rows)
Jim Carol Dan Lisa
Content 4 3 3 3
Structure 3 3 2 4
Grammar 2 3 2 2
Total 9 9 7 9
Assessment Examples
Assessment Activity/Assessment Measure
Need to address two components: Assessment activity – what will
students do to show you they have achieved the SLO
Assessment measure – how will instructors evaluate what the students have done
Assessment Activity Examples
Licensing Exams (e.g., Nursing) Standardized Tests Reflective Self-Assessment Essay Satisfaction/perception surveys
(student, faculty, staff, employer, community)
Case Study & Problem Solving
Use an “in situ” approach to simulate real life situations and problems.
Flowchart or Diagram
Visual/graphic illustration of a process or system.
High level cognitive achievement requiring analysis and synthesis.
“Draw a flowchart for whatever you do. Until you do, you do not know what you are doing, you just have a job.” (W.E. Deming, quality guru)
Capstone
Capstone = a culminating event or crowning achievement
Capstone courses/projects
Portfolios
Portfolios are a collection of student work over a period of time, usually including student reflection on their achievement.
Have strengths/weaknesses; ask yourself if it will work for you
ePortfolios
Assessment Measures
Checklist Rubric Calibrated Peer Review
Checklists
Determines whether a criterion is present or not. Good for simple psychomotor skills or low level recall.
Example: Hand Washing Checklist
Adjusted water temp
Hands wetted
Soap applied
Lather worked up
Applied cleansing friction >20 sec
Applied friction between fingers
Applied friction back of hands
Rinsed of all soap
Dried appropriately
Rubrics
A rubric is "a scoring tool that lists the criteria for a
piece of work or 'what counts.' " (Heidi Goodrich)
describes levels of quality for each of the criteria, usually on a point scale
makes your expectations clear to students. reduces the time you spend grading student
work and makes it easier for you to explain to students why they got the grade they did and what they can do to improve
are most effective when you provide students with actual examples of poor, average, and good work
Calibrated Peer Review
“Calibrated Peer Review (CPR)™ is a Web-based program that enables frequent writing assignments even in large classes with limited instructional resources. In fact, CPR can reduce the time an instructor now spends reading and assessing student writing.”
“CPR offers instructors the choice of creating their own writing assignments or using the rapidly expanding assignment library. Although CPR stems from a science-based model, CPR has the exciting feature that it is discipline independent and level independent.”
Assessing Program-Level SLOs
Licensing/Employment/Transfer Capstone Courses or Projects Student Surveys Portfolios
Create an Assessment Tool
Look at the SLOs for your course. Are there any assignments that
provide good data on outcomes? If not, you need to create one!
Create an Assessment Tool
Determine which type of assessment tool would best assess that students can DO the outcome
Should be authentic – closely resembling a real life experience
Will the student perform a task, create a product, analyze a case study, solve a problem?
Identify the Purpose of the Assessment
Will it be formative or summative? If formative – how will feedback be
given? If summative – will the student have
ample practice and feedback to do what is expected?
What is a Successful Outcome?
Identify the major traits that determine a successful outcome
Describe the criteria relating to the traits and create a checklist, rubric or set of descriptive performance standards
Set criteria at the appropriate level of thinking (Bloom’s taxonomy)
Create an Assessment Tool
Try out your assessment on student work and make appropriate modifications.
Share the tool with other faculty and get feedback.
Online Resources
Calibrated Peer Review: http://cpr.molsci.ucla.edu
ePortfolios: http://eportfolio.org, http://www.osportfolio.org
Hot Potatoes: http://hotpot.uvic.ca/ Rubrics: http://rubistar.4teachers.org.
http://landmark-project.com/ rubric_builder/index.php, http://rubrics.coastline.edu, http://school.discoveryeducation.com/schrockguide/assess.html
Online Resources
Internet Resources for Higher Education Outcomes Assessment: http://www2.acs.ncsu.edu/UPA/assmt/resource.htm
CLOSING THE LOOP
The Student Learning Outcomes Assessment Cycle
Develop, modify, or review a curriculum, course, program, or service.
Develop Student Learning Outcomes (SLOs)
Design & Measure Student Learning as a result of the Curriculum, Course, or Program.
Collect, discuss, and analyze data.
Determine refinements based on data. Closing the Assessment Loop
The Assessment Cycle (SLOAC)
Reporting the SLOAC
Goal – to assess every course and program in your discipline within the five-year program review cycle.
Annual reporting and Program Review reporting.
Report includes: SLO and assessment methods used Assessment Results How results were used for improvement
of the course or program
The Paper Trail
Course and Program SLO forms need a Department Approval form.
Submit to Erline Ewing in Academic Affairs (for VCCC approval).
Other areas submit to area coordinator: Student Services – Walter Jones Administrative Services – Brick Durley President’s Office – Cherine Trombley