Page 1

COMPETENCY BASED ASSESSMENT

Page 2

Contents• Terminologies

• What is CBE?

• Characteristics of CBE

• Elements of CBE

• Competency based assessment

• Differences between traditional and competency based assessment

• Types of CBA approaches

Page 3

What is Competency based education?

Competency based education focuses on learner performance often called learning outcomes in reaching specific goals and objectives of the curriculum. 

Page 4

How do we look at CBE?

Learning Teaching

Educational Outcomes

Page 5

What is competence?

• Competence equating to performance – that is the ability to perform nursing tasks

• a psychological construct – that is the ability to effectively integrate cognitive, affective and psychomotor skills when delivering nursing care.

Page 6

What is competency?

•K- KNOWLEDGE•S- SKILLS•A- ABILITIES

Page 7

Characteristics of CBE• Explicit & aligned with expected

competencies.• criteria driven and focus on

accountability• is grounded in real life experiences• Fosters learner’s ability to self assess• Individualized, providing more

opportunity for independent study.

Page 8

Competency based assessment• Competency assessment in nursing

includes demonstration of theoretical content knowledge , technical skills and also assessment of a candidate’s attitudes and practice.

Page 9

Designing Competency based assessment process1. Identify units of competency

2. Develop competency profile

3. Identify evidence requirements

4. Review possible evidence gathering techniques

5. Select the appropriate technique

6. Prepare the evidence plan

7. Document the evidence

8. Review the assessment process

Page 10

Differences between TA & CBA

Traditional assessment

• Theoretical• Curricular outcomes

defined by college/board

• Integrated with learning

• Norm referenced

CBA

• Actual performance• Outcomes are

determined by experts/research

• Independent of learning program

• Criterion referenced

Page 11

Tools for assessment of competence• Assessment of competence must

include more than one indicator• Technical skills can be assessed by

standardized tools such as Nurse competence scale

• Non technical skills can be assessed using paper and pencil tests, observational checklists and patient feedback or patient satisfaction.

Page 12

Indicators for competence in clinical nursing practice• portfolios • examinations • peer review (assessment) • direct observation • self-assessment • interview • patient outcomes

Page 13

PORTFOLIOS• Portfolios are a collection of evidence

to demonstrate skills, knowledge, attitudes and achievements.

• Portfolios may be used as a personal development tool or an assessment tool.

• Portfolios rely on a level of self-regulation, writing and critical reflection skills.

Page 14

• Consider the need for clarity, defined limits and adequate preparation of individuals and assessors.

• Confidence in the use of portfolios for both students and assessors will only develop with time and experience.

Page 15

Evidences of portfolioPatient profile/community profile Notes of care givenReflective account of a critical practice

incident Review of an individual patient’s needs

and care Citation of published literature to justify

particular aspect of care

Page 16

Tips to use Portfolio in CBA • the evaluation tool should be clear and specific• a tripartite approach- student, mentor, assessor • assessors should be trained• consider formative assessment for reflective

components• consider a summatively assessed clinical skills list • provide opportunities for assessors to network with

each other to share experiences and encourage a consistent approach

Page 17

Objective structured clinical examination• An OSCE is a series of stations/exercises

through which students rotate individually to demonstrate a range of skills and knowledge.

• OSCEs provide the opportunity for teaching and assessment.

• OSCEs are costly and can be time-consuming.

• Adequate preparation of participants, assessors and actors is important.

Page 18

Peer Assessments• Peer assessment/review provides a

mutually supportive approach for ensuring accountability and fostering professional growth.

• peer review is often regarded as a mechanism for demonstrating professionalism and promoting professional development.

Page 19

Who is a peer?• a colleague of equal status in the same

ward

• a colleague of equal status from another ward in the same specialty/department

• someone of higher status in the same ward or another ward in the same specialty

• someone of equal or higher status from another hospital.

Page 20

Benefits of peer review• confirmation of previously held beliefs in an

individual’s skills/lack of certain skills

• provides clarity to self-assessment findings

• provides a medium for sharing colleagues’ experiences

• helps to identify problems

• provides an opportunity for feedback on performance.

Page 21

Issues with peer review• Assessment is dependent on the reviewer’s

expertise and familiarity with the individual’s work and role expectations

• can be a source of anxiety for both parties

• issues around negative comments

• risk of feedback being overcritical or judgmental

• can cause disagreement among peers

• depends on availability of peers or of opportunities for peer assessment.

Page 22

Recommendations to use peer review in CBA• Use peer review in combination with portfolios to encourage self-reflection prior to seeking critical comment.

• Define terms and scoring criteria and provide training for reviewers.

• Reviewers should be familiar with the individual’s role expectations and relevant standards.

Page 23

Direct Observation• ‘observation of nursing activities in the clinical

setting instead of a contrived examination format’.

Issues highlighted for consideration in relation to direct observation include:

• the need for accuracy and consistency of assessment

• awareness of the range of variables that can influence practice in the clinical setting

• the role of personal characteristics.

Page 24

Factors causing bias in direct observation • Familiarity

• the assessor favoring or disfavoring the person being assessed

• nervousness of the person being assessed • resource deficiencies with which an

external assessor is unfamiliar • educators who are not sufficiently

involved in practice• successful performance on one day is no

guarantee of it on another.

Page 25

Self assessment• Most common and most favored in

nursing competence surveys• Formalised method of self monitoring

and self reflection• cost-effectiveness • gives individuals conscious control over

practice.• Helps nurses to identify own strengths

in practice and the weaknesses.

Page 26

Patient outcomes• Patient oucomes and patient views are

very valuable in assessment of competence.

• Patients may find it difficult to differentiate between the caring aspect and technical aspect

• Anxiety over nurses response to competence assessment

• May not separate caring and judging

Page 27

Tips to use CBA• Timing and environment must be conducive to

assessment

• It should be a supportive process that is beneficial rather than punitive.

• It is part of the developmental process

• It must allow for the provision of supportive feedback

• Link with appropriate opportunities for professional development

Page 28

• Recognize and minimize bias in the assessment process.

• Develop different methods of assessment for different domains of competence

• Observational contexts must be limited, defined and calibrated .

• Adequate training in the field of competency assessment to assessors

• Reliable competency assessment processes.

Page 29

Criticisms of CBA • Difficulties and inconsistencies in implementing

non-graded assessment systems:

• Inconsistencies in judgments between assessors.

• Limited understanding of the design and implementation of assessment tools.

• Difficulty in separating summative purposes of assessment with formative purposes.

Page 30

Key points:• Competence is a complex concept and

debates continue about the most appropriate definition and method of assessment.

• In broad terms, competency standards recommend expected levels of knowledge, attitudes, skills and behaviors.

• A key challenge in competence assessment is ensuring objectivity.

• •

Page 31

• Assessment of competence should use more than one indicator.

• Regardless of the tool or indicator, it is essential that there is adequate preparation of the individual being assessed, the assessors and mentors.

• It has been suggested that the first step in competence assessment should be self-reflection.

Page 32

ConclusionNursing as a profession is constantly under assault

and variety of other roles encroach upon the nursing roles. The competency movement has contributed to this continuing debate. The distinguishing aspects of nursing, the ways in which it can be developed and implemented by its practitioners, and the recognition of what is unique in nursing are the features that have been enhanced by the scrutiny. All nurse educators should be encouraged to explore and implement the innovative assessment approaches such as CBA and OBA in order to ensure that nursing education in India is on par with the developed countries.

Page 33

References1. Lenburg, C. Redesigning Expectations for Initial and Continuing

Competence For Contemporary Nursing Practice Online Journal of Issues in Nursing. 1999. Available http://www.nursingworld.org /ojin

2. Artis, LH. Identifying core competencies for the Army Nurse. Research paper, US Army Command and General Staff College; Kansas 2003.

3. Standards Of Professional Nursing Practice. American Nurses Associatio(ANA). Scope and Standards of Nursing Practice, 2004

4. Entry to Practice Competencies For Ontario Registered Nurses as of January 1, 2005.College of Nurses of Ontario. Pub. No. 41037; 2004

5.

Page 34

5. Common Competencies for Registered Nurses. WPSEAR: ANMC Australia; March 2006

6. Practice Standards for Nurses in India. Teaching Material for Quality Assurance Model: Nursing. Indian Nursing Council, New Delhi.2006

7 Accreditation Council for Graduate Medical Education. ACGME Outcome Project. 2005. www.acgme.org/Outcome

Page 35

Page 36

MRS HEMALATHA R

PROFESSOR AND PRINCIPAL

UNIVERSAL COLLEGE OF NURSING

OUTCOME BASED ASSESSMENT

1

Page 37

CONTENTS

• What is outcome based education (OBE)?

• Why do we choose OBE?

• What instructional strategies go with OBE?

• How to design assessment tasks in line with OBE?

2

Page 38

Outcome based approach

• an emphasis on the results of learning (outcomes);

• a focus on learning by doing;

• a focus on what learners can do as well as learning of content;

• opportunities for the recognition of prior learning

3

Page 39

outcomes

• Outcomes are more detailed and specific statements derived from the goals.

• What you want the end result of your efforts to be.

It is not what you are going to do to the student

• Use active verbs such as articulate, illustrate, conduct, synthesize, analyze, etc.

• Terms: construct, locate, dissect, categorize, compose, invent…..

4

Page 40

Abilities

• Identify and solve problems

• Work in a team

• Organise and manage

• Collect, analyse and evaluate information

• Communicate effectively

• Use science and technology

• Recognise problem solving contexts

5

Page 41

outcome statements • Fuzzy: Students will understand and apply

Principles of pediatric nursing in different situations

• In between: Students can articulate how principles of pediatric nursing can help in different situations.

• Precise: Students will describe how applying Pediatric nursing principles can facilitate change in the care of Ms Liza , a terminally ill child.

6

Page 42

Outcome statements

1. Students will be able to compare and contrast various leadership styles.

2. Students will be able to identify when students are cheating.

3. Students will be able to locate all of the items they need for a balanced diet.

4. Students will demonstrate care when using the hospital equipment.

7

Page 43

What is OBE?

Normal teaching:

What topics do I teach?

• OBE:What do I intend my students to be able to

do after my teaching that they could not do before, and to what standard?

8

Page 44

What is OBE?

Normal teaching:

What teaching methods do I use?

OBE:

How do I supply learning activities that will help them achieve those outcomes?

9

Page 45

Normal teaching:• Teaching is conceived as a process of

transmitting contents to the students.

OBE:• It is stated clearly not what the lecturer

is going to teach, but what the outcome of that teaching is intended to be and at what standard.

10

Page 46

Principles of obe

1. Clear goals

2. Standard referenced

3. Learning for all

11

Page 47

Why OBE?

• To address mismatches between employers and graduates.

• Students main concerns are GRADES, employers are looking for KNOWLEDGE, ATTITUDE and SKILLS.

12

Page 48

Why obe?

• Tells students what they are supposed to learn, how and to what standard.

• OBE addresses both declarative and functional knowledge

13

Page 49

T-l activities in obe

We need Teaching and Learning activities that require students to

• Apply

• Invent

• Generate new ideas

• Diagnose and solve problems

14

Page 50

TEACHING LEARNING ACTIVITIES

• Problem-based learning

• Cooperative learning

• Blended learning

• Discovery learning

15

Page 51

What is expected of students?

• Do challenging tasks other than memorize and reproduce what was taught.

• Write project proposals, complete projects, analyse case studies, give case presentations, show their abilities to think, question, research, and make decisions based on the findings.

16

Page 52

Traditional assessment OB assessment

• Continuous testing & permanent grading

• Mistakes on permanent record: best grades & records fast & consistent performers; slower learners never catch up

• Never assess/ document what learners can ultimately do successfully

• Macro view learning & achievement

• Mistakes inevitable steps in development, internalizing & demonstrating high level of performance capabilities

• Ultimate achievement what able to do

Traditional vs ob assessment17

Page 53

Features of ob assessment

• EMPHASIS ON OUTPUT• CRITERION REFERENCED• CONTINUOUS• IS BOTH FORMATIVE AND SUMMATIVE• IS VALID• INTEGRATED• ALIGNED WITH PURPOSES OF THE

EDUCATIONAL PROGRAM

18

Page 54

Assessment tasks

Not how well students have received knowledge,

but

how they can use it in academically and professionally appropriate ways such as

• Solving problems

• Designing experiments

• Communicating with clients

19

Page 55

Designing Assessment Tasks• The assessment tasks (AT) must be

aligned to the learning outcomes. Eg in PBL

• The grading criteria, or rubrics,are the key to easy and successful grading. Final grades depends on how well all the outcomes have been achieved.

20

Page 56

OB ASSESSMENT PROCESS

• Begin with your outcomes• Design/select your assessment task• Identify appropriate sections of the assessment task• Set criteria statements• Take each statement and identify the categories

where you require evidence to be produced• Take each category and describe what you would

expect a “highly competent” candidate to produce in each of these categories

• Do the same for a “competent” candidate• Do the same for a “not yet competent” candidate

21

Page 57

Measurement methods

Direct

Indirect

22

Page 58

Another way of looking at it

• Naturally occurring methods – writing essay, observations, student interactions, student debates

• Purposefully designed methods- surveys

23

Page 59

DIRECT ASSESSMENT METHODS

• Student work samples• Collections of student work (e.g. Portfolios)• Capstone projects, dissertations, projects• Project-embedded assessment• Observations of student behavior• Internal juried review of student projects• External evaluations of student performance• Document analysis (e.g., clinical diary)

24

Page 60

DIRECT ASSESSMENT METHODS

• External juried review of student projects• Externally reviewed internship• Performance on a case study/problem• Performance on problem and analysis (Student• explains how he or she solved a problem)• Performance on national licensure examinations• Locally developed tests• Standardized tests• Pre-and post-tests• Essay tests blind scored across units

25

Page 61

Indirect methods

• Alumni, Employer, Student Surveys• Exit Interviews with Graduates• Graduate Follow-up Studies• Percentage of students who go on to graduate school• Retention and Transfer Studies• Job Placement Statistics• Faculty/Student ratios• Percentage of students who study abroad• Enrollment trends• Percentage of students who graduate within five-six

years

26

Page 62

Assessment in obe

• Outcomes are set out sequentially on a gradation of increasing complexity that students are expected to master.

• OBE focuses on how much and how well the students have learnt.

• Weaker students may have to follow a different learning path & finish later.

27

Page 63

Clarifying grades

• Grades may…

Collapse information about several learning outcomes

Reflect other aspects of student performance

Student 1 Student2 Student3

Outcome 1 A B A

Outcome 2 B B C

Outcome 3 A C C

28

Page 64

Tips for teachers

• Educators must reduce their reliance on written tests and exam papers;

• Educators must increase their range of assessment styles;

• These styles must be appropriate to the outcomes educators are assessing;

• Assessment should be continuous and integrated into teaching.

29

Page 65

30Operation Models for OBE

Yr. 1

Yr. 4

Yr. 3

Yr. 2

K 70%

S&A30%

K 70% K 70% K 70%

S&A30%

S&A30%

S&A30%

Distribution of K, S, A elements throughout the 4 years

A B C D A B C D

Page 66

Take home messages

• You do not have to assess everything you do every year.

• You don’t have to do everything at once-start with 2 or 3 learning outcomes

• Think baby steps• Be flexible• Acknowledge and use what you have already done.• Borrow examples from other institutions to modify

as appropriate• Time for this must be re-allocated• Allocate time according to priorities

31

Page 67

conclusion

• The focus of education has shifted from the educator to learner however this shift requires change within the educational system in order to facilitate learning. Establishing an Outcome based assessment system for education is the best way for a particular learner to reach the desired outcomes. The role of the educator is to enable and encourage all learners to achieve essential outcomes while the learner actively participates in and contributes towards the learning process.

32

Page 68

resources

• Killen, R. Outcomes-Based Education: Principles and Possibilities. Unpublished manuscript. University of Newcastle, Australia: Faculty of Education. Retrieved September 11, 2004 from www.schools.nt.edu.au

• A brief guide to outcomes based assessment. Academic development. Rhodes University.2009

• Bresciani, M.J. The relationship between outcomes, measurement. and decisions for continuous improvement. National Association for Student Personnel Administrators, Inc NetResults E-Zine ; September, 2002.

• Palomba, C.A. and Banta, T.W. Assessment• essentials: Planning, implementing and improving• assessment in Higher Education. San Francisco:

Jossey-• Bass;1999.

33

Page 69

• THANK YOU