Assessment in anOutcomes-based Education

Carlo Magno, PhD.Correspondence: crlmgn@yahoo.com

Objectives Construct a one

session plan showing outcomes with appropriate assessment and delivery mode.

Case A Cherry is a staff in Mall X. Her work is to get the

products in the stock room as requested by the sales lady. Cherry has been working in Mall X for two months right after her graduation. Cherry finished a course in nursing and she is expected to assist doctors and care for patients . One time the HR called her and asked her to clean the wound of another employee and put some bandage. She asked for an alcohol and an automizer spray. She placed the alcohol inside the automizer and started spraying on the open wound. The employee started to cry because the terrible pain. She shouted at the patient “stop crying like a baby! This will hurt more if you don’t stop!”

Case B Cheryl conducted a graduate tracer study to

determine if the nursing program of the school is adequate. She found in the survey that: 100% of the graduates were employed as nurses

in different sectors. 100% of the graduates were able to get a job in

two months. 100% of the graduates passed the licensure exam

for nursing. 60% of the faculty are receiving a salary of less

than Php15,000 80% of them working as a nurses were rated

satisfactory in their hospital performance by the hospital administrators.

Is the nursing program adequate?

Outcomes-based Education Clearly focusing and organizing everything in

an educational system around what is essential for all students to be able to do successfully at the end of their learning experiences.

This means starting with a clear picture of what is important for students to be able to do, then organizing the curriculum, instruction and assessment to make sure this learning ultimately happens (Spady, 1994).

Outcomes-based Education In the process of designing program curriculum,

the outcomes of the learning is emphasized and pre-determined

What is expected from the learning after the students have graduated in order to equip them with the necessary skills and capabilities before they enter the work place

Then go backward with: curriculum design programme outcomes and course outcomes, development of instructions delivery modes appropriate assessments methodologies

Outcomes-based Education looking at the level at which the inputs,

methods, and execution produce the desired learning competencies for the graduates of that program as determined by the Technical Committees/Technical Panels and as measured by appropriate assessments.

It points to the way in which the level of attainment of the outcomes can be progressively heightened.

Outcomes-based Education Assessing student growth and competency in

relation to these outcomes Detailing how outcomes based learning at a

whole program level functions within a complex university context

What needs to be set? Mission and Vision are translated into: indicators (e.g., professional qualification or

employability) metrics (e.g., percentage of passing in a

licensure exam or percentage of employment) targets (e.g., 70% passing or 85% employed).

Outcomes-based Education Outcomes – what learners are expected to know and

be able to do at the desired level of competence Outcomes-based evaluation – clearly focusing and

organizing everything in an educational system around what is essential for all students to be able to do successfully at the end of the learning experiences.

Outcomes-based teaching and learning – constructive alignment of intended, learning outcomes with appropriate outcomes-based assessment methods and teaching and learning activities. OBE applied in the classroom level.

Performance criteria – specific, measureable statements identifying the performance(s) required to meet the outcome; conformed through evidence.

Outcomes-based Education Ensure quality assurance (QA) 1) to translate vision, mission, and goals

(VMG) into desired learning outcomes 2) to establish the proper learning

environment (implementation of teaching-learning systems as well as support processes and procedures)

3) to review against performance indicators and standards defined in the assessment system

4) to enhance programs and systems

Approaches on OBE A direct assessment of educational outcomes,

with evaluation of the individual programs that lead to those outcomes. (To make sure that outcomes are delivered)

An audit of the quality systems of an institution, to determine whether these are sufficiently robust and effective to ensure that all programs are well designed and deliver appropriate outcomes. (To deliver effective programs)

What needs to be established? Mission and vision Program Educational Objectives Program Outcomes Matrix of courses with program outcomes

(Curriculum map) Outcome-based teaching and learning

delivery system Program assessment and evaluation process Continuing quality improvement program

ExampleLearning outcome Indicators Assessment

At the end of the course students should be able to:• decide which

inferential statistics can be used for a specific hypothesis

• Encode data acceptable in SPSS

• use SPSS to compute for the inferential statistics

• Suggest which type of statistics to be used given a hypothesis

• Use SPSS to encode survey data

• Click appropriate menu in SPSS when computing for ANOVA, t-test etc.

• Test: Given a hypothesis, write the correct stats to be used

• Performance based: encode the data from a survey to the SPSS worksheet

• Checklist: step by step procedure in using SPSS

How do you teach in an OBE framework? Transformative learning Learner-centered Understanding by Design

Case presentation

A group of nursing students at the start of the term thinks that doing research is going to the library and compiling information about a topic from different books. The teacher knew about it by asking the students at the start of the class what research is. The teacher started to show examples of journal articles and how research is conducted. The different methodologies and the use of research was assigned. Then the students conducted their own research using a nonexperimental design. Towards the end of the term when the teacher asked again what is research, students see it as a process of arriving at new knowledge and supporting the ideas through data (evidence).

Assessment What is the function of assessment? How do you help students learn better

through assessment?

Types of Assessment Selected Response

Binary Choices Multiple Choice Matching Type

Constructed Response “Supply Test” Short Form answers - identification Completion – fill in the blanks, cloze test Essay

Alternative Forms Performance-based Authentic-based Portfolio Assessment

Features of performance assessment Intended to assess what it is that students know

and can do with the emphasis on doing. Have a high degree of realism about them

(authentic). Involve: (a) activities for which there is no single

correct answer, (b) assessing groups rather than individuals, (c) testing that would continue over an extended period of time, (d) self-evaluation of performances.

Likely use open-ended tasks aimed at assessing higher level cognitive skills.

Performance assessment Bring testing

methods more in line with instruction.

Assessment should approximate closely what it is students should know and be able to do.

Emphasis of performance assessment Should assess higher

level cognitive skills rather than narrow and lower level discreet skills.

Direct measures of skills of interest.

Characteristics of performance-based assessment Students perform, create, construct, produce, or do

something. Deep understanding and/or reasoning skills are needed

and assessed. Involves sustained work, [often days and weeks]. Calls on students to explain, justify, and defend. Performance is directly observable. Involves engaging in ideas of importance and substance. Relies on trained assessor’s judgments for scoring Multiple criteria and standards are prespecified and

public There is no single correct answer. If authentic, the performance is grounded in real world

contexts and constraints.

Learning Targets Skills

Communication and presentation skills Ex: Speaking1. Speaking clearly, expressively, and audibly

a. Using voice expressivelyb. Speaking articulately and pronouncing words

correctlyc. Using appropriate vocal volume

2. Presenting ideas with appropriate introduction, development, and conclusion

1. Presenting ideas in an effective order2. Providing a clear focus on the central idea3. Providing signal words, internal summaries, and

transitions

3. Developing ideas using appropriate support materials

a) Being clear and using reasoning processesb) Clarifying, illustrating, exemplifying, and documenting

ideas

4. Using nonverbal cuesa. Using eye contact b. Using appropriate facial expressions, gestures, and

body movement

5. Selecting language to a special purposea. Using language and conventions appropriate for the

audience

Psychomotor skills Fine motor: cutting papers with scissors, drawing a

line tracing, penmanship, coloring drawing, connecting dots

Gross motor: Walking, jumping, balancing, throwing, skipping, kicking

Complex: Perform a swing golf, operate a computer, drive a car, operate a microscope

Visual: Copying, finding letters, finding embedded figures, identifying shapes, discrimination

Verbal and auditory: identify and discriminate sounds, imitate sounds, pronounce carefully, blend vowels

Products Write promotional materials Report on a foreign country Playing a new song

Variation of authenticity

Relatively authentic

Somewhat authentic

Authentic

Identify the materials used in dressing a wound

Give the steps in dressing a wound

Dresses the wound of a patient

Tell the use of a thermometer

Records temperature in a chart

Uses the thermometer on a patient and records it.

Explain the steps on taking blood pressure

Show how to take the blood pressure using a sphygmomanometer

Get the blood pressure of a patient.

Constructing Performance Based tasks1. Identify the performance task in which

students will be engaged2. Develop descriptions of the task and the

context in which the performance is to be conducted.

3. Write the specific question, prompt, or problem that the student will receive.

• Structure: Individual or group?• Content: Specific or integrated?• Complexity: Restricted or extended?

Complexity of task Restricted-type task

Narrowly defined and require brief responses Task is structured and specific Ex:

Construct a bar graph from data provided Demonstrate a shorter conversation in French about what

is on a menu Read an article from the newspaper and answer questions Flip a coin ten times. Predict what the next ten flips of the

coin will be, and explain why. Listen to the evening news on television and explain if you

believe the stories are biased. Construct a circle, square, and triangle from provided

materials that have the same circumference.

Extended-type task Complex and elaborate Often include collaborative work with small group

of students. Requires the use of a variety of information Examples:

Design a playhouse and estimate cost of materials and labor

Plan a trip to another country: Include the budget and itinerary, and justify why you want to visit certain places

Conduct a historical reenactment (e. g. impeachment trial of ERAP)

Diagnose and repair a car problem Design an advertising campaign for a new or existing

product

Identifying Performance Task Description Prepare a task description Listing of specifications to ensure that

essential if criteria are met Includes the ff.:

Content and skill targets to be assessed Description of student activities

Group or individual Help allowed

Resources needed Teacher role Administrative process Scoring procedures

Performance-based Task Question Prompt Task prompts and questions will be based on

the task descriptions. Clearly identifies the outcomes, outlines what

the students are encourage dot do, explains criteria for judgment.

Example of a task Prompt:

Characteristics of Tasks

1. Should integrate the most essential aspects of the content being assessed with the most essential skills.

2. Should be authentic Realistic Require judgment and innovation Ask the student to do the subject Replicates or stimulates Assess the students ability to efficiently and

effectively use a repertoire of knowledge and skill to negotiate a complex task

Allows opportunities to rehearse, practice, consult resources, and get feedback and refine performances and products.

3. Structure the task to assess multiple learning targets

4. Structure the task so that you can help students succeed.

5. Think through what students will do to be sure that the task is feasible

6. The task should allow for multiple solutions7. The task should be clear8. The task should be challenging and stimulating to

students9. Include explicitly stated scoring criteria as part of

the task10. Include constraints in completing the task

Performance Criteria What you look for in student responses to

evaluate their progress toward meeting the learning target.

Dimensions of traits in performance that are used to illustrate understanding, reasoning, and proficiency.

Start with identifying the most important dimensions of the performance

What distinguishes an adequate to an inadequate demonstration of the target?

Questions to ask: What are the attributes of good writing, or

good scientific thinking, or good collaborative group process, of effective oral presentation? More generally, by what qualities or features will I know whether students have produced an excellent response to my assessment task?

What do I expect to see if this task is done excellently, acceptably, or poorly?

Questions to ask: Do I have samples or models of student work,

from my class or other sources, that exemplify some of the criteria I might use in judging this task?

What criteria for this or similar task exist in my national curriculum framework, my state assessment program, my district curriculum guides, my school assessment program?

What dimensions might I adapt from work done by natural curriculum councils, by other teachers?

Example of Criteria Learning target:

Students will be able to write a persuasive paper to encourage the reader to accept a specific course of action or point of view.

Criteria: Appropriateness of language for the audience Plausibility and relevance of supporting

arguments. Level of detail presented Evidence of creative, innovative thinking Clarity of expression Organization of ideas

Rating Scales Indicate the degree to which a particular

dimension is present. Three kinds: Numerical, qualitative, combined

qualitative/quantitative

Numerical Scale Numbers of a continuum to indicate different level

of proficiency in terms of frequencyor quality

Example:Complete Understanding 5 4 3 2 1 No understanding

Clear organization 5 4 3 2 1 No organization

Fluent reader 5 4 3 2 1 Emerging reader

Qualitative scale Uses verbal descriptions to indicate student

performance. Provides a way to check the whether each

dimension was evidenced. Type A: Indicate different gradations of the dimension Type B: Checklist

Example of Type A: Minimal, partial, complete Never, seldom, occasionally, frequently, always Consistent, sporadically, rarely None, some, complete Novice, intermediate, advance, superior Inadequate, needs improvement, good excellent Excellent, proficient, needs improvement Absent, developing, adequate, fully developed Limited, partial, thorough Emerging, developing, achieving Not there yet, shows growth, proficient Excellent, good, fair, poor

Example of Type A: Checklist

Holistic scale The category of the scale contains several criteria,

yielding a single score that gives an overall impression or rating

Examplelevel 4: Sophisticated understanding of text indicated with constructed meaninglevel 3: Solid understanding of text indicated with some constructed meaninglevel 2: Partial understanding of text indicated with tenuous constructed meaninglevel 1: superficial understanding of text with little or no constructed meaning

Example holistic scale

Analytic Scale One in which each criterion receives a separate

score.

Example

Criteria Outstanding5 4

Competent 3

Marginal2 1

Creative ideas

Logical organization

Relevance of detail

Variety in words and sentences

Vivid images

Rubrics When scoring criteria are combined with a

rating scale, a complete scoring guideline is produced or rubric.

A scoring guide that uses criteria to differentiate between levels of student proficiency.

Example of a rubric

Rubrics should answer the following questions: By what criteria should performance be judged? Where should we look and what should we look for

to judge performance success? What does the range in the performance quality

look like? How do we determine validity, reliability, and fairly

what scores should be given and what that score means?

How should the different levels of quality be described and distinguished from one another?

Workshop Create a one session plan that will run for

one session. Indicate one learning outcomes

(performance-based) Provide the indicators How will you assess the outcomes? Provide the following: Nature of the final

product, what students are required to do, instrument (rubric/checklist)