Post on 05-Dec-2014
description
transcript
Assessment in anOutcomes-based Education
Carlo Magno, PhD.Correspondence: crlmgn@yahoo.com
Objectives Construct a one
session plan showing outcomes with appropriate assessment and delivery mode.
Case A Cherry is a staff in Mall X. Her work is to get the
products in the stock room as requested by the sales lady. Cherry has been working in Mall X for two months right after her graduation. Cherry finished a course in nursing and she is expected to assist doctors and care for patients . One time the HR called her and asked her to clean the wound of another employee and put some bandage. She asked for an alcohol and an automizer spray. She placed the alcohol inside the automizer and started spraying on the open wound. The employee started to cry because the terrible pain. She shouted at the patient “stop crying like a baby! This will hurt more if you don’t stop!”
Case B Cheryl conducted a graduate tracer study to
determine if the nursing program of the school is adequate. She found in the survey that: 100% of the graduates were employed as nurses
in different sectors. 100% of the graduates were able to get a job in
two months. 100% of the graduates passed the licensure exam
for nursing. 60% of the faculty are receiving a salary of less
than Php15,000 80% of them working as a nurses were rated
satisfactory in their hospital performance by the hospital administrators.
Is the nursing program adequate?
Outcomes-based Education Clearly focusing and organizing everything in
an educational system around what is essential for all students to be able to do successfully at the end of their learning experiences.
This means starting with a clear picture of what is important for students to be able to do, then organizing the curriculum, instruction and assessment to make sure this learning ultimately happens (Spady, 1994).
Outcomes-based Education In the process of designing program curriculum,
the outcomes of the learning is emphasized and pre-determined
What is expected from the learning after the students have graduated in order to equip them with the necessary skills and capabilities before they enter the work place
Then go backward with: curriculum design programme outcomes and course outcomes, development of instructions delivery modes appropriate assessments methodologies
Outcomes-based Education looking at the level at which the inputs,
methods, and execution produce the desired learning competencies for the graduates of that program as determined by the Technical Committees/Technical Panels and as measured by appropriate assessments.
It points to the way in which the level of attainment of the outcomes can be progressively heightened.
Outcomes-based Education Assessing student growth and competency in
relation to these outcomes Detailing how outcomes based learning at a
whole program level functions within a complex university context
What needs to be set? Mission and Vision are translated into: indicators (e.g., professional qualification or
employability) metrics (e.g., percentage of passing in a
licensure exam or percentage of employment) targets (e.g., 70% passing or 85% employed).
Outcomes-based Education Outcomes – what learners are expected to know and
be able to do at the desired level of competence Outcomes-based evaluation – clearly focusing and
organizing everything in an educational system around what is essential for all students to be able to do successfully at the end of the learning experiences.
Outcomes-based teaching and learning – constructive alignment of intended, learning outcomes with appropriate outcomes-based assessment methods and teaching and learning activities. OBE applied in the classroom level.
Performance criteria – specific, measureable statements identifying the performance(s) required to meet the outcome; conformed through evidence.
Outcomes-based Education Ensure quality assurance (QA) 1) to translate vision, mission, and goals
(VMG) into desired learning outcomes 2) to establish the proper learning
environment (implementation of teaching-learning systems as well as support processes and procedures)
3) to review against performance indicators and standards defined in the assessment system
4) to enhance programs and systems
Approaches on OBE A direct assessment of educational outcomes,
with evaluation of the individual programs that lead to those outcomes. (To make sure that outcomes are delivered)
An audit of the quality systems of an institution, to determine whether these are sufficiently robust and effective to ensure that all programs are well designed and deliver appropriate outcomes. (To deliver effective programs)
What needs to be established? Mission and vision Program Educational Objectives Program Outcomes Matrix of courses with program outcomes
(Curriculum map) Outcome-based teaching and learning
delivery system Program assessment and evaluation process Continuing quality improvement program
ExampleLearning outcome Indicators Assessment
At the end of the course students should be able to:• decide which
inferential statistics can be used for a specific hypothesis
• Encode data acceptable in SPSS
• use SPSS to compute for the inferential statistics
• Suggest which type of statistics to be used given a hypothesis
• Use SPSS to encode survey data
• Click appropriate menu in SPSS when computing for ANOVA, t-test etc.
• Test: Given a hypothesis, write the correct stats to be used
• Performance based: encode the data from a survey to the SPSS worksheet
• Checklist: step by step procedure in using SPSS
How do you teach in an OBE framework? Transformative learning Learner-centered Understanding by Design
Case presentation
A group of nursing students at the start of the term thinks that doing research is going to the library and compiling information about a topic from different books. The teacher knew about it by asking the students at the start of the class what research is. The teacher started to show examples of journal articles and how research is conducted. The different methodologies and the use of research was assigned. Then the students conducted their own research using a nonexperimental design. Towards the end of the term when the teacher asked again what is research, students see it as a process of arriving at new knowledge and supporting the ideas through data (evidence).
Assessment What is the function of assessment? How do you help students learn better
through assessment?
Types of Assessment Selected Response
Binary Choices Multiple Choice Matching Type
Constructed Response “Supply Test” Short Form answers - identification Completion – fill in the blanks, cloze test Essay
Alternative Forms Performance-based Authentic-based Portfolio Assessment
Features of performance assessment Intended to assess what it is that students know
and can do with the emphasis on doing. Have a high degree of realism about them
(authentic). Involve: (a) activities for which there is no single
correct answer, (b) assessing groups rather than individuals, (c) testing that would continue over an extended period of time, (d) self-evaluation of performances.
Likely use open-ended tasks aimed at assessing higher level cognitive skills.
Performance assessment Bring testing
methods more in line with instruction.
Assessment should approximate closely what it is students should know and be able to do.
Emphasis of performance assessment Should assess higher
level cognitive skills rather than narrow and lower level discreet skills.
Direct measures of skills of interest.
Characteristics of performance-based assessment Students perform, create, construct, produce, or do
something. Deep understanding and/or reasoning skills are needed
and assessed. Involves sustained work, [often days and weeks]. Calls on students to explain, justify, and defend. Performance is directly observable. Involves engaging in ideas of importance and substance. Relies on trained assessor’s judgments for scoring Multiple criteria and standards are prespecified and
public There is no single correct answer. If authentic, the performance is grounded in real world
contexts and constraints.
Learning Targets Skills
Communication and presentation skills Ex: Speaking1. Speaking clearly, expressively, and audibly
a. Using voice expressivelyb. Speaking articulately and pronouncing words
correctlyc. Using appropriate vocal volume
2. Presenting ideas with appropriate introduction, development, and conclusion
1. Presenting ideas in an effective order2. Providing a clear focus on the central idea3. Providing signal words, internal summaries, and
transitions
3. Developing ideas using appropriate support materials
a) Being clear and using reasoning processesb) Clarifying, illustrating, exemplifying, and documenting
ideas
4. Using nonverbal cuesa. Using eye contact b. Using appropriate facial expressions, gestures, and
body movement
5. Selecting language to a special purposea. Using language and conventions appropriate for the
audience
Psychomotor skills Fine motor: cutting papers with scissors, drawing a
line tracing, penmanship, coloring drawing, connecting dots
Gross motor: Walking, jumping, balancing, throwing, skipping, kicking
Complex: Perform a swing golf, operate a computer, drive a car, operate a microscope
Visual: Copying, finding letters, finding embedded figures, identifying shapes, discrimination
Verbal and auditory: identify and discriminate sounds, imitate sounds, pronounce carefully, blend vowels
Products Write promotional materials Report on a foreign country Playing a new song
Variation of authenticity
Relatively authentic
Somewhat authentic
Authentic
Identify the materials used in dressing a wound
Give the steps in dressing a wound
Dresses the wound of a patient
Tell the use of a thermometer
Records temperature in a chart
Uses the thermometer on a patient and records it.
Explain the steps on taking blood pressure
Show how to take the blood pressure using a sphygmomanometer
Get the blood pressure of a patient.
Constructing Performance Based tasks1. Identify the performance task in which
students will be engaged2. Develop descriptions of the task and the
context in which the performance is to be conducted.
3. Write the specific question, prompt, or problem that the student will receive.
• Structure: Individual or group?• Content: Specific or integrated?• Complexity: Restricted or extended?
Complexity of task Restricted-type task
Narrowly defined and require brief responses Task is structured and specific Ex:
Construct a bar graph from data provided Demonstrate a shorter conversation in French about what
is on a menu Read an article from the newspaper and answer questions Flip a coin ten times. Predict what the next ten flips of the
coin will be, and explain why. Listen to the evening news on television and explain if you
believe the stories are biased. Construct a circle, square, and triangle from provided
materials that have the same circumference.
Extended-type task Complex and elaborate Often include collaborative work with small group
of students. Requires the use of a variety of information Examples:
Design a playhouse and estimate cost of materials and labor
Plan a trip to another country: Include the budget and itinerary, and justify why you want to visit certain places
Conduct a historical reenactment (e. g. impeachment trial of ERAP)
Diagnose and repair a car problem Design an advertising campaign for a new or existing
product
Identifying Performance Task Description Prepare a task description Listing of specifications to ensure that
essential if criteria are met Includes the ff.:
Content and skill targets to be assessed Description of student activities
Group or individual Help allowed
Resources needed Teacher role Administrative process Scoring procedures
Performance-based Task Question Prompt Task prompts and questions will be based on
the task descriptions. Clearly identifies the outcomes, outlines what
the students are encourage dot do, explains criteria for judgment.
Example of a task Prompt:
Characteristics of Tasks
1. Should integrate the most essential aspects of the content being assessed with the most essential skills.
2. Should be authentic Realistic Require judgment and innovation Ask the student to do the subject Replicates or stimulates Assess the students ability to efficiently and
effectively use a repertoire of knowledge and skill to negotiate a complex task
Allows opportunities to rehearse, practice, consult resources, and get feedback and refine performances and products.
3. Structure the task to assess multiple learning targets
4. Structure the task so that you can help students succeed.
5. Think through what students will do to be sure that the task is feasible
6. The task should allow for multiple solutions7. The task should be clear8. The task should be challenging and stimulating to
students9. Include explicitly stated scoring criteria as part of
the task10. Include constraints in completing the task
Performance Criteria What you look for in student responses to
evaluate their progress toward meeting the learning target.
Dimensions of traits in performance that are used to illustrate understanding, reasoning, and proficiency.
Start with identifying the most important dimensions of the performance
What distinguishes an adequate to an inadequate demonstration of the target?
Questions to ask: What are the attributes of good writing, or
good scientific thinking, or good collaborative group process, of effective oral presentation? More generally, by what qualities or features will I know whether students have produced an excellent response to my assessment task?
What do I expect to see if this task is done excellently, acceptably, or poorly?
Questions to ask: Do I have samples or models of student work,
from my class or other sources, that exemplify some of the criteria I might use in judging this task?
What criteria for this or similar task exist in my national curriculum framework, my state assessment program, my district curriculum guides, my school assessment program?
What dimensions might I adapt from work done by natural curriculum councils, by other teachers?
Example of Criteria Learning target:
Students will be able to write a persuasive paper to encourage the reader to accept a specific course of action or point of view.
Criteria: Appropriateness of language for the audience Plausibility and relevance of supporting
arguments. Level of detail presented Evidence of creative, innovative thinking Clarity of expression Organization of ideas
Rating Scales Indicate the degree to which a particular
dimension is present. Three kinds: Numerical, qualitative, combined
qualitative/quantitative
Numerical Scale Numbers of a continuum to indicate different level
of proficiency in terms of frequencyor quality
Example:Complete Understanding 5 4 3 2 1 No understanding
Clear organization 5 4 3 2 1 No organization
Fluent reader 5 4 3 2 1 Emerging reader
Qualitative scale Uses verbal descriptions to indicate student
performance. Provides a way to check the whether each
dimension was evidenced. Type A: Indicate different gradations of the dimension Type B: Checklist
Example of Type A: Minimal, partial, complete Never, seldom, occasionally, frequently, always Consistent, sporadically, rarely None, some, complete Novice, intermediate, advance, superior Inadequate, needs improvement, good excellent Excellent, proficient, needs improvement Absent, developing, adequate, fully developed Limited, partial, thorough Emerging, developing, achieving Not there yet, shows growth, proficient Excellent, good, fair, poor
Example of Type A: Checklist
Holistic scale The category of the scale contains several criteria,
yielding a single score that gives an overall impression or rating
Examplelevel 4: Sophisticated understanding of text indicated with constructed meaninglevel 3: Solid understanding of text indicated with some constructed meaninglevel 2: Partial understanding of text indicated with tenuous constructed meaninglevel 1: superficial understanding of text with little or no constructed meaning
Example holistic scale
Analytic Scale One in which each criterion receives a separate
score.
Example
Criteria Outstanding5 4
Competent 3
Marginal2 1
Creative ideas
Logical organization
Relevance of detail
Variety in words and sentences
Vivid images
Rubrics When scoring criteria are combined with a
rating scale, a complete scoring guideline is produced or rubric.
A scoring guide that uses criteria to differentiate between levels of student proficiency.
Example of a rubric
Rubrics should answer the following questions: By what criteria should performance be judged? Where should we look and what should we look for
to judge performance success? What does the range in the performance quality
look like? How do we determine validity, reliability, and fairly
what scores should be given and what that score means?
How should the different levels of quality be described and distinguished from one another?
Workshop Create a one session plan that will run for
one session. Indicate one learning outcomes
(performance-based) Provide the indicators How will you assess the outcomes? Provide the following: Nature of the final
product, what students are required to do, instrument (rubric/checklist)