The Design and Implementation of Outcome-Based Assessment
in Science
Teaching Excellence Seminar Teaching Excellence Seminar School of ScienceSchool of Science
University of Science and TechnologyUniversity of Science and Technology
22 March 201322 March 2013
Isabella Wai-Yin Poon Isabella Wai-Yin Poon
[email protected]@cuhk.edu.hk
Department of StatisticsDepartment of Statistics
Associate Dean of ScienceAssociate Dean of Science
Director of Centre for Learning Enhancement & ResearchDirector of Centre for Learning Enhancement & Research
The Chinese University of Hong KongThe Chinese University of Hong Kong
2
Content
1. Introduction
2. An OBA Assessment Model
3. Assessment as Learning Activities
4. Evaluation of Outcome at Programme Level
5. Recent Initiatives at Science of CUHK
3
Introduction
4
Outcome-based Approach (OBA) OBA
Actively promoted by UGC since 2005 Weave into the (2012) 4-year curriculum
Many professors have no knowledge on this UGC
Not straightjacket Institutions should take ownership Inter-institutional Task Force established
5
OBA: An International Trend? http://www.cwsei.ubc.ca
6
OBA Development @ CUHK General consensus at CUHK
Clearly articulated learning outcomes Curriculum components align with learning
outcomes
• Content
• Learning activities
• Assessment Assessing outcomes and collecting feedback
7
How We Proceeded? OBA has long been used
implicit rather than explicit manner These concepts are part of our culture
Although we are not eloquent in using the language
1st step: articulate programme-level learning outcomes Outcomes developed in 2006
A series of TDG projects (faculty- or department-levels) to implement initiatives that address issues of our concern Front-line initiatives carry the spirit of OBA Formulate in an OBA framework
8
Science Teaching: Areas for improvement
Interaction in class Variety of learning activitiesCultivate the development of learning outcomes
Generic skills Value/attitude
Assessment Align with OBA? Summative vs Formative assessment
• Summative assessment: Provides evidence of student attainment• Formative assessment: Assessment for learning
Criterion- vs norm-referencing• Criterion: Pegged to pre-determined or expected learning otucomes• Norm: Defined by relative performance
Various TDG or other projects Faculty, programme or individual level
9
Faculty-level TDG Project 1
“Engaging Science Students in the Design and Engagement of Assessment” October 2005 – November 2008 Website: http://www.cuhk.edu.hk/sci/TDG/
10
Faculty-level TDG Project 2 “Promoting the Use of the Outcome-Based
Approach to Science Programmes: Development of Learning Outcome Statements and Measurement Instruments” October 2006 – November 2008 Website: http://www.cuhk.edu.hk/sci/OBA/
11
Faculty-level TDG Project 3
“Using Clickers to Enhance Teaching and Learning: Pilot Case Studies in Science Programmes” December 2008 – October 2009 Website: http://www.cuhk.edu.hk/sci/clicker
12
Faculty-level TDG Project 4
The Development of an OBA Assessment Model for Science Subjects” August 2009 to Dec 2012 Website: will soon launch
13
Faculty-level TDG Project 5 “Inter-faculty Academic-related Advising Service”
November 2009 – Website: Restricted access (CUHK students only) http://serverfarm.elearningpro.com/cuhk/softskillmentor
14
Faculty-level TDG Project 6 “Learning by Engaging and Teaching”
January 2013 – Website: http://www.cuhk.edu.hk/cpse/LET/
15
An OBA Assessment Model
16
Norm- vs criterion-referencing Heated debate
Science: Norm-referenced approach CUHK 2008 QAC Audit Report (p.23)
“There is therefore an argument that criterion-referencing as opposed to norm referencing is more consistent with OBA. The Panel encourages CUHK to engage with the pressure placed by OBA on norm-based grading and consider the place of criterion based assessment and the measures which may be introduced, other than norm referencing, to track and correct grade inflation”
Similar statements found in the QAC Audit reports for other UGC-funded institutions
HKBU, 2009, p.18; HKU, 2009, p.22; CITYU, 2010, p.21
17
Two Components
The Assessment Framework How does a course, and in particular its
assessment, support the development of programme LOs ?
The Assessment Chart Develop the grade descriptors Use both norm referencing and criterion
referencing
18
Assessment Framework
19
Assessment Chart
… Assessment Chart
20
21
Characteristics of the Model User-friendly Align assessment with outcomes Scaffold the planning and design of
assessment (components) Communicate clear expectation to students A probabilistic but not a deterministic model
22
A probabilistic but not a deterministic model
Age
Individual
Expected
Height
xy ˆˆˆ
line Regression
23
A Project:Assessment as a Learning Activity
24
Objective
Use assessment as a learning activity More formative assessment
Promote a variety of assessment activities Don’t rely too much on examination
Increase students’ interest in the learning activities
Enhance fairness in assessment
25
Two Strategies
Strategy 1: Students participate in the design of assessment criteria
Strategy 2: Students conduct self- and peer-assessments based on the set of criteria Method introduced to ensure students to seriously
and critically conduct self- and peer-assessments
26
Strategy 1: Assessment Criteria
Ask students to develop the set of criteria through a rigorous process All criteria are proposed by students
• Too time consuming
Provide students with a set of criteria for students’ deliberation and decision Easier and more efficient
27
… Strategy 1: Assessment
Criteria Ownership of the assessment criteria
Students critically think about what constitute quality work
Students know in advance the set of criteria Pay attention to various aspects
• Some aspects may otherwise be ignored Writing skills in addition to content Presentation skills
o Use of IT, tables, graphso Time control
Enhance transparency and fairness on assessment activities
28
Strategy 2: Self- and peer-assessment Self-assessment is graded based on
whether students can assess themselves properly
29
… Strategy 2: Self- and peer-assessments Improve fairness in peer- & self-assessments Students must pay attention to peer’s work in
order to assess themselves properly More critical More serious Learn from peers
Issues to note Benchmark the self- and peer-assessment % of self-assessment contributes to total score More discussion in a paper*
*Poon, W.Y., McNaught, C., Lam P., & Kwan H.S. (2009). Improving assessment methods in university science education with negotiated self- and peer-assessment. Journal of Assessment in Education: Principles, Policy & Practice, 16:3, 331-346.
30
Experiences in Science Used by many science teachers in
Group presentation Individual presentation Group project written report Individual written assignment Laboratory experiment report Poster presentation In-class activities in student-oriented courses
• Debate
• Role-play
• Portfolio of activities designed to explore a topic
31
Evaluation Examples of students’ positive comments
“Quite interesting and innovative.” “Innovative, can allow students to identify and think about
what really matters.” “This can force students to pay attention to the
presentations of the others instead of daydreaming or chatting to each other.”
“It allows students to be more aware of their performance towards their peers, such that the presentation is more interesting and interactive.”
“It helps us to focus on the aspects other than the content itself, like the presentation skills and how the questions from audience are answered. Through this process, we could learn much more from each other.”
32
…Evaluation
Examples of students’ suggestions “As an undergraduate student, it is a bit difficult for
us to assess other’s work.” “Students may not have enough knowledge to
comment other’s group topic.” “A weighting can be assigned to each marking
criterion to better reflect the importance of each criterion.”
“There may be strong disagreement among group members in deciding the scores … I think the assessment will be much fairer if it is carried out individually.”
33
Teachers’ Views
In general well-received by teachers Also Students Easy to use Challenges in classes of large size
Applicable to a variety of activities Students paying more attention to peer’s works Can address the issue of free-riders in group
project Promote the use of group project
• as an effective assessment method
• as learning activity
34
…Teachers’ Views
Most effective for assessing generic skills Some students still prefer teacher’s assessment
• Content knowledge or professional skills Ask students to peer- and self-assess generic skills but
not content or professional skills Can nurture the development of a wide range of (skill-
and value-domain) outcomes Self-managed learning, life-long learning, team-work,
active learning, critical thinking, honesty, integrity…
Evaluation of Outcome at Programme Level
Student Experience Questionnaire (SEQ)
35
Student Experience Questionnaire Administer to all undergraduate programmes @
CUHK 3-year cohort: 1st & final year 4-year cohort: 2nd & final year
Two parts, 16 constructs, 33 questions Part 1: Students’ Capability Development
7 constructs, 14 questions Part 2: Teaching & Learning Environment
9 constructs, 19 questions All questions are on 5-point scale
from (1) strongly disagree to (5) strongly agree
36
… Student Experience Questionnaire The 7 constructs for “Capability Development”
Critical thinking Creative thinking Self-managed learning Adaptability Problem solving Communication skills Interpersonal skills and groupworks
37
… Student Experience Questionnaire The 9 constructs for “Teaching and Learning
Environment” Active learning Teaching for understanding Feedback to assist learning Assessment Relationship between teachers and students Workload Relationship with other students Cooperative learning Coherence of curriculum
38
Areas for Attention Capability
Communication skills Interpersonal skills and groupwork
Teaching & Learning Environment Active learning
• My teaching staff use a variety of teaching methods
• We are given the chance to participate in class Workload
TDG projects developed to improve in these areas
39
… Science SEQ Results (2003 – 2012)
Improvement observed Communication skills Active learning
Many factors contribute to the improvement Many front-line teaching and learning
initiatives
40
41
Recent Initiatives at the Science Faculty at CUHK
42
Project Information: Book & Website Published “Recent
Initiatives in Teaching and Learning: Faculty of Science 2010” Overview a number
of T&L projects
43
Website: http://www.cuhk.edu.hk/sci/ritl
44
45
More details for each project
46
Q & A
Thank you very much.