Assessment and evaluation

Prof. Namir G. Al-TawilHawler Medical University

College of Medicinenamiraltawil@yahoo.com

Contents

• Objectives• Definitions of assessment and evaluation• Types of assessment– Formative assessment– Summative assessment

• Assessment of students’ knowledge• Assessment of students’ performance

Objectives

• At the end of the lecture, the audience must be able to know:

• The differences between formative and summative assessment.

• How to assess students’ knowledge.• How to assess students’ performance.

Definitions

• Assessment– It is the process of data gathering by instructors

about their teaching and their students’ learning.– Pre-test, observations, examinations are forms of

assessment.• Evaluation– It is the decision making process based on the

assessment data.

Types of assessments

• Diagnostic assessment• Formative assessment• Summative assessment

Diagnostic assessment

• Helps to identify students’ current knowledge, skills, and capabilities; and to clarify misconception before teaching takes place.

• Types:– Pre-tests– Self assessment– Interview

Formative assessment

• Provides feedback during a teaching process.• It assesses student’s and teacher’s progress.• These assessments are typically not graded.• E.g: if after 2 weeks of the start of the course,

you made a test (by clickers for e.g) and you discovered that the majority didn’t answer correctly, then you have to repeat this point, and to correct yourself in the next course

Types of formative assessment

• Observations during in-class activities.• Homework exercises.• Question and answer sessions (planned and

spontaneous).• Conferences between instructor and students.• Student feedback about their instructors.

Summative assessment

• Takes place at the end of learning process.• It assesses the final product.• Grades are usually an outcome of it.– To decide whether the student is eligible for the

next step or not.

Types of summative assessments

• A mid-term exam.• A final exam.• A final project.• A paper.

Criteria of the assessment tool

• Validity• Reliability• Educational impact, on teaching and learning• Acceptability to staff, students, and others• CostA variety of assessment methods are needed to

test competency

“When the cook tastes the soup, that’s formative evaluation. When the guests taste the soup, that’s summative evaluation”

Bob Stake 1991

Assessment of knowledge and its application

• The most common method for knowledge assessment is the written method.

• Several formats are available like MCQs and essays….

• The content of the question is more important than the format.

• A variety of formats are preferable.

Questions formatsMCQs (A-Type: one best answer)

• Single choice questions are more preferred than the true/false (multiple response item).

• Anatomy of the item (question):– Stem• Vignette (e.g case study)• Lead-in question (what is the most likely Dx?)

– Options• Correct answers• Distracters

Tips for designing MCQs

• Use Plausible Distracters (wrong-response options)

• Use a Question Format (rather than incomplete sentence format)

• Keep Option Lengths Similar• Balance the Placement of the Correct Answer• Be Grammatically Correct

• Avoid Clues to the Correct Answer• Avoid Negative Questions• Use Only One Correct Option • Give Clear Instructions• Avoid the "All the Above" Option• Avoid the "None of the Above" Option• Don't Use MCQs When Other Item Types Are

More Appropriate

MCQs (R-type: Extended matching items)

• Extended matching questions consist of lettered options followed by a list of numbered problems/questions. For each numbered problem/question select the one lettered option that most closely answers the question. You can use the lettered options once, more than once, or not at all.

Key features questions

• They are short clinical cases or scenarios which are followed by questions aimed at key features or essential decisions of the case.

• The questions could be MCQs or open ended questions.

• More than one correct answer could be provided.

Short answers questions

• Open ended questions.• Their answer is usually 1 or 2 words.• Usually used when closed formats are not

possible.• Each teacher should correct the same

question for all the students. So more reliable scores.

Essay questions

• Requires more time to answer.• Less reliable scoring.• Using correction scheme (checklist) can

improve reliability.

Modified essay questions

• Is a special type of essay question that consists of a case followed by a series of questions that relate to the case and that must be answered in the sequence asked.

• A student answering the first question incorrectly is likely to answer the subsequent questions incorrectly too.

Assessment of performance

• Categorized into:– In vitro assessment (using simulation)– In vivo assessment (real condition)

• In both, the student make a skill or behavior and observed by the examiner.

Assessment of performance, cont

• Many methods like check-lists and rating scales can be used to report observation.– OSCE and OSPE– Direct observation of procedural skills (DOPS)– Peer assessment– Self-assessment

Methods for assessing performance

• Checklists– E.g OSCE

• Rating scales– Problem of subjectivity and low reliability– Use more than one rater (observer)

• OSCE• Short cases to assess clinical competence

• Long cases (to assess clinical competence)– No more used in many colleges– Of help for formative examinations– Modification done in many places to improve

reliability• 360 degrees evaluation– Of help in formative assessment– Multiple evaluators like supervisors, peers,

students, administrative staff, and patients.– Time consuming and not practical.

• Mini Clinical Evaluation Exercises (MINI-CEX)– Of help in formative assessment– They are relatively short observations (15-20

minutes) in which performance is recorded on a 4 point scale where 1 is unacceptable, 2 is below expectation, 3 is met expectations, and 4 is exceeded expectations.

– Several competencies are evaluated: history taking, physical examination, clinical judgment, counseling, professionalism and other generic qualities

• Portfolio– A portfolio is a collection of student work which

provides evidence that learning has taken place.– Mainly for formative assessment.

Questions

• ?• ?• ?

Conclusion

• Various assessment methods that test a range of competencies are available for examiners

• The choice depends on purpose of examination, i.e. either summative, formative or both, in addition to other factors.

Thanks for listening