Competency#based Learning in Traumatology

David Hill, MS, FRCS, FRCS(Ed), FRACS, Paul Stalley, MBBS, FRACS, David Pennington, MBBS, FRCS(Ed), FRACS, Michael Besser, MBBS, FRCSC, FRACS,

William McCarthy, MEd, FRACS, Sydney, Australia

BACKGROUND: A multidisciplinary, competency- based trauma teaching program was introduced for final year medical students (n = 67) at Royal Prince Alfred Hospital (RPAH) in 1994 to comple- ment the surgical clerkship.

METHODS: The method involved small groups ro- tating through a series of teaching stations each structured to address a predetermined compe- tency. Four 3-hour sessions were held on the subject areas of resuscitation, plastic, ot-thope- die, and neurotrauma.

Performance in the trauma section of a summa- tive Objective Structured Clinical Examination (OSCE), 6 months after the teaching, was com- pared with that of a control group (n = 127) from other campuses where trauma was taught by a series of discipline-based lectures. Three trauma OSCE stations were designed to test psychomo- tor skills while five addressed aspects of the cog- nitive domain. Checklists were used to ensure standardization of scoring in a range of questions asked or skills tested at each station.

RESULTS: The marks of the RPAH students (mean 78% f SD 9%) were significantly higher (P < 0.0005) than the controls (mean 70% -c SD 9%) in the 8 trauma questions. There was no significant difference (P = 0.8) in marks obtained by the study group (mean 61% ? SD 8%) and controls (mean 63% & SD 7%) in 22 questions sampling a wide spectrum of nontrauma subject areas. The study group performed significantly better in one of the three skills stations and three of the five problem-solving stations when compared with the control group.

CONCLUSION: The innovation has the potential to fulfill a need for an integrated trauma pro- gram in the undergraduate core curriculum. Am J Surg. 1997;173:136-140. 0 1997 by Excerpta Medica, Inc.

From the Department of Surgery Education Centre, The Univer- sity of Sydney and Royal Prince Alfred Hospital, Sydney, Australia.

This work was supported by a National Teaching Development Grant, Committee for Advancement of University Teaching, Can- berra, Australia

Requests for reprints should be addressed to Dr. David Hill, Royal Prince Alfred Hospital, Missenden Road, Camperdown, NSW, 2050, Australia

Manuscript submitted March 9, 1995 and accepted in revised form October 2, 1995.

M ajor changes are occurring in the process of edu- cating medical students. There is a worldwide swing from the traditional to competency-based

systems of medical education. A catalyst for such change has been the advocacy of such bodies as the World Feder- ation for Medical Education.’

While the Advanced Trauma Life Support (ATLS) pro- gram* is a successful, competency-based teaching system for use at the postgraduate level, a need exists for an appropri- ate model to use in medical schools. Lucas et al have ad- dressed this need for a national trauma core curriculum and have proposed a method based on a hierarchy of compe- tencies in the cognitive, psychomotor, and attitudinal do- mains.’ Trauma related subjects were rated highly for in- clusion in the surgical curriculum in several discipline-based surveys in the areas of general4 thoracic, 5 and plastic sur- gery.6

A multidisciplinary, competency-based system to teach trauma care to medical students has been developed at Royal Prince Alfred Hospital (RPAH). The SCORPIO model7 that was initially used to teach initial assessment and resuscitation s has been expanded to encompass the definitive and rehabilitative phases of management.

It was felt that such a strategy would be more effective than a discipline-based trauma lecture program to produce a graduate with the knowledge, attitudes, and skills to man- age patients with major injury during the early post-grad- uate years.

This article describes the teaching methods used and pre- sents the results of a competency-based assessment of stu- dents exposed to the two different teaching methods.

METHODS The teaching innovation was developed at RPAH, one of

the four principal teaching hospitals of the University of Sydney. The multidisciplinary trauma teaching program was introduced into the undergraduate surgical curriculum at RPAH in 1992, the year that the State Trauma Plan was implemented in Sydney. Under this plan, all four teaching hospitals of the university were designated trauma centers, three Level I and one Level II. RPAH was designated as a Level I center. The innovation was phased in partially re- placing the traditional, discipline-based trauma lecture pro- gram that has continued to be used on the other three cam- puses as the sole method of formal instruction. Sixty-seven medical students at RPAH took part in the program, both as Year IV students in 1992 and final-year (Year VI) stu- dents in 1994. Final-year students on all four campuses com- pleted a similar 15week surgical clerkship program. This clerkship included obligatory rotations in neurosciences

136 0 1997 by Excerpta Medica, Inc. 0002-961 O/97/$1 7.00

All rights reserved. PII SOOO2-9610(96)00412-6

(3 weeks), orthopedics/trauma (3 weeks), cardiothoracic surgery (1 week), and anesthetics (1 week).

Teaching The SCORPIO trauma sessions were based on a series of

modules, each incorporating a study guide, the teaching, and formative assessment. The study guides’ defined the competencies to be acquired and how they are best achieved. They were distributed to both students and tutors at RPAH at an appropriate time before the teaching. The content of the trauma study guides was abstracted from the Department of Surgery curriculum handbook but was pre- sented in an integrated rather than discipline-based format. This handbook, available to students and staff on all cam- puses, outlines the content and objectives of the surgical core curriculum.

The teaching sessions, each lasting 2’/1 hours, were held once for Year IV students in 1992 and 4 times during April 1994 for the same student cohort then in their final year. A 30-minute assessment immediately followed each teach- ing session. This took the form of a Multiple Choice Ques- tionnaire (MCQ), a Group-Objective Structured Clinical Examination (G- OSCE), or a series of Structured Short Answer (SSA) questions. An evaluation of these formative assessment methods has been reported previously.”

SCORPIO is a medium for competency-based teaching. The philosophy of the method is inherent in the acronym, which stands for teaching that is structured, clinical, objec- tive-referenced, problem-based, integrated, and organized. The mechanics involve the whole of the final year, being divided into small groups. Each group rotates through a se- ries of 6 teaching stations, spending 25 minutes at each. A teacher at each station conducts an interactive tutorial ad- dressing a particular competency: a problem solving exer- cise, a clinical skill, or a procedural skill.

The session on Initial Assessment and Resuscitation of the injured patient was run for the 1992, Year IV students and repeated in 1994 for the same students then in their final year. The subject matter covered at the six stations is shown in Table I(a). The competencies addressed at Year IV level focused on factual knowledge, clinical assessment, and prin- ciples of resuscitation. These were assessed by pre/post-test MCQ. The teaching focus changed in final year to address a higher level of competency including problem-solving ex- ercises and the necessary skills for airway control, pleural decompression, intravenous access, peritoneal lavage, and extremity immobilization. Competence was assessed on completion of the session by G-OSCE. Teaching was mul- tidisciplinary with tutors drawn from anesthetics, intensive care, emergency medicine, and surgery. A prerequisite to teach on this session was having an ATLS certificate or its Australian equivalent (EMST). Full details of this module and the results of its evaluation have been previously re- ported.’

The Department of Plastic and Reconstructive Surgery conducted the second Year VI trauma session covering the subject matter listed in Table I(b). There were two stations on wounds: one focusing on the physiology of wound heal- ing and the other involving the students in practical aspects of wound closure. There were two stations on the hand: one focusing on the diagnosis of hand injury and the second on practical aspects of splintage and rehabilitation. The station

TABLE I

Subject Material Presented at the Four SCORPIO Sessions

(a) Initial Assessment and Resuscitation

Airway management Chest injury Shock

Head injury Abdominal injury Extremity injury

(b) Plastic Surgery/ Trauma

Wounds-biology

Wounds-suturing Hand injury-

assessment Hand injury-definitive

care Facial injuries-

assessment

Burns-management

(c) Neurotrauma Intracranial hematoma-definitive

care

Cerebral edema-definitive care Spinal injury-definitive care Rehabilitation-head injury

Rehabilitation-spinal cord injury Glasgow Coma Scoring

(d) Orthopedics/Trauma Fractures and/or dislocations

of the: Pelvis

Femur Knee Leg/ankle Shoulder, humerus, elbow

Forearm and wrist

COMPETENCY-BASED LEARNING IN TRAUMATOLOGY/HILL ET AL

on facial injuries was structured to teach a system for ex- amining the facial skeleton, the orbital contents, and the upper aerodigestive tract. The burns station focused on re- suscitation and aspects of wound management. Students were assessed by four SSA questions immediately on com- pletion of this session.

The third session in the Year VI series was structured to portray a multidisciplinary approach to neurotrauma. The subject areas covered are listed in Table I(c). Neurosur- geons conducted the stations on the definitive care of in- tracranial hematoma and spinal injury while an intensive care physician conducted the station on cerebral edema. A neuropsychologist presented a real patient with cognitive impairment for discussion at the station on head injury re- habilitation while a physician in rehabilitative medicine demonstrated the positive aspects of function in patients with spinal cord injury. A standardized patient was used to teach Glasgow Coma Scoring. Formative assessment was again by problem-oriented SSA questions.

The final, Year VI session was conducted by the Depart- ment of Orthopedic Surgery. The session systematically covered the spectrum of extremity Injury listed in Table I(d). The principal objectives of the session were to develop the skills of clinical assessment, interpretation of radiolog- ical studies, and factual knowledge concerning definitive treatment. Real and standardized patients were used as re- sources to achieve the first mentioned objective. Again stu- dents were assessed on completion by SSA questions.

Assessment One hundred and ninety-four students from the 4

teaching hospitals underwent a 30-station Objective Structured Clinical Examination (OSCE)“,” in October 1994. The OSCE was held simultaneously at four loca- tions using identical questions, standardized resources, and checklists for marking. This assessment was 6 months after the RPAH students completed their structured trauma teaching program. Eight of the OSCE stations were related to trauma (Table II), while 22 sampled a

THE AMERICAN JOURNAL OF SURGERY@ VOLUME 173 FEBRUARY 1997 137

‘OMPETENCY-BASED LEARNING IN TBAUMATOLOGY/HILL ET AL

TABLE II

Trauma OSCE Stations

Topic Resources Competency Assessed

Primaly survey multi-trauma Standardized patient Physical examination

Basic airway support Bag and mask Clinical procedure ventilator-Manikin

Pneumothorax Tube thoracostomy Clinical procedure set-Manikin,

chest x-ray Shock X-ray-Fractured Patient management problem

pelvis, femur, ankle Epidural hematoma CT scan Patient management problem Eye injury Written vignette Patient management problem Scaphoid fracture X-ray Patient management problem Tendon injury (hand) Photograph Patient management problem

TABLE Ill

Summative Assessment (OSCE)

OSCE Study Group Control Group

(n = 67) (n = 127) P-Value

Trauma questions

(n = 8)

Nontrauma questions (n = 22)

Mean 78% SD 9%

Mean 61%

SD 8%

Mean 70% SD 9%

Mean 63%

SD 7%

<0.0005

0.8

TABLE IV

Student Performance (Mean Marks) at the Individual Trauma OSCE Stations

Study Group Control Group (n = 67) (n = 127)

Clinical Skills Stations Mean (%) Mean (%) P Value

Primary survey 74 73 0.5 Airway support 79 80 0.6 Tube thoracostomy 85 70 <O.OOl

Patient Management Stations

Shock 80 81 0.7 Epidural hematoma 72 55 <O.OOl

Eye injury 82 70 <O.OOl Scaphoid fracture 77 74 0.1 Hand iniurv 73 62 0.01

wide range of surgical and medical topics. The OSCE questions were compiled and marked by faculty from all four campuses. The authors of this paper were not in- volved in marking at any stations.

A typical OSCE station consisted of a written vignette based on a standardized patient, a manikin, or other vi- sual resource. Students were required to demonstrate a clinical skill or respond to a series of open-end, short answer questions, usually related to patient management. Stations were structured in a standard fashion to ensure that all students were asked to respond to identical ques- tions posed in the same format. Checklists were used by examiners at each station detailing content to be covered and marks to be given for each subsection of a station. Five minutes was allocated to complete each station.

An example of one of the trauma clinical skills stations consisted of a short vignette about a patient brought to Emergency after having fallen down a flight of stairs. The

student is required to identify several fractured ribs and a pneumothorax on a chest radiograph. The candidate must then select the appropriate intercostal catheter and dem- onstrate the sequential steps of tube thoracostomy on a manikin. A checklist details the marks to be allocated for performance at each component of the station.

An example of a nontrauma OSCE station used in the assessment concerns a patient with a pancreatic pseudocyst. A written vignette detailing the clinical features is pre- sented in conjunction with a computerized tomographic scan showing the pseudocyst. Marks were allocated for each of five open-ended questions concerning: radiologic diag- nosis, complications, cyst contents, treatment options, and advice to prevent recurrence.

The performance by the 67 RPAH students in the 8 trauma questions was compared with that of the 127 students at the 3 hospitals where trauma was taught by traditional methods. Performance in the 22 nontrauma

138 THE AMERICAN JOURNAL OF SURGERY@ VOLUME 173 FEBRUARY 1997

1 COMPETENCY-BASED LEARNING IN TRAUMATOLOGY/HILL ET AL 1

OSCE questions was compared with the 2 groups as a control.

Statistical Methods The Student’s t test was used to evaluate the statistical

significance of differences in mean performance between the study group (RPAH) and the controls (other campuses) in both the trauma and the nontrauma OSCE questions.

RESULTS The study group who took part in the SCORPIO teach-

ing program performed significantly better (P < 0.0005) in the eight trauma OSCE questions (mean 78% t SD 9%) when compared with that of the control group (mean 70% ? SD 9%) taught in the traditional manner.

There was no significant difference in performance be- tween the study and control groups in the 22 nontrauma OSCE questions. These data are summarized in Table III.

The marks obtained by the two groups at the individual trauma stations are shown in Table IV. The study group skills performance at the thoracostomy station was sig- nificantly better than that of the control group. There was no significant difference between the groups in skills performance at the primary survey and airway support stations.

The marks obtained by the study group were significantly higher than those of the control group in three of the five stations testing cognitive skills. This superior performance in questions relating to patient management concerned the subject areas of head, hand, and ocular injury. A distinction level performance was demonstrated by both groups at the shock station with the mean marks not differing signifi- cantly. There was no significant difference in performance at the scaphoid fracture station.

DISCUSSION The worldwide trauma epidemic continues unabated

producing a loss of productive life greater than cancer, heart disease, and stroke combined.13 It is imperative that trauma be given a high priority at the levels of under- graduate, postgraduate, and continuing medical educa- tion. Over a decade has passed since Trunkey commented on the deficiencies of medical school trauma education.14 These deficiencies require correction by a system of trauma teaching comparable with the highly successful ATLS program.

The short-term goal of an effective undergraduate trauma program is to produce a graduate with the knowledge, con- fidence, and skills to assess, resuscitate, and initiate defini- tive care of an injured patient. A longer term goal of such a program is to enhance trauma care outcomes. A system of sound trauma education at medical school continuing into the postgraduate years is felt to be a key factor to achieve this latter goal.

This article is more about the process of teaching rather than content. The challenge is how to best present trauma to the students in a form representing the multidisciplinary nature of contemporary clinical practice rather than as a series of didactic, discipline-based lectures. The traditional lecture is a medium to present factual material to a large group. The lower cognitive domain is usually the only com- petency addressed. The SCORPIO system provides a me-

dium to actively engage small groups in acquiring higher cognitive, clinical, and procedural skills. Each teaching sta- tion is structured to address a defined competency adding a dimension not possible in the traditional lecture. Simula- tion using standardized patients” and manikins is used fre- quently at the stations, obviating the difficulties of using real patients for teaching acute surgery. These strategies in- corporate many of the concepts of contemporary medical educational practice.i*i6

The desirable content of a trauma curriculum has been amply defined in a series of discipline-based stud- ies.4-6*‘7,‘s The aim of the SCORPIO program was to take the content advocated on a discipline basis and to pres- ent this in the form of an integrated, multidisciplinary educational package. Integration of content was achieved with competencies addressed in the spectrum of care ranging from resuscitation through to definitive manage- ment and rehabilitation. Integration of teaching was achieved with tutors drawn from nine involved clinical departments.

Formative assessment (FA) is regarded as an important component of the learning process.‘” Previous work has shown that the use of FA on the SCORPIO program is favorably regarded by the students.‘” Its use has proved to be a valuable strategy to guide the learning process.” Three FA modalities were used on the trauma program. The G-OSCE” was a valuable technique to assess team performance, which is regarded as a desirable feature of trauma practice. The SSA questions” proved to be an appropriate modality to assess problem-solving skills, while the MCQ is a traditional way to assess the recall of factual information.

Obligatory clerkships in the neurosciences, orthopedics/ trauma, anesthetics, and cardiothoracic surgery were com- mon on all four campuses. These are regarded as integral components of the trauma learning experience that allow students to apply the skills acquired during their formal teaching to real life clinical situations.

The OSCE was used to assess performance in a summative setting at the end of the undergraduate course. The eight OSCE trauma questions used in this assessment addressed a range of problem-solving and clinical and procedural skills with a planned bias in the area of assessment and resusci- tation The performance by the study group in the trauma questions was significantly better than that of the controls. This enhanced performance was seen particularly in the patient management questions and to a lesser extent at the clinical skills stations.

The integrated program of competency-based trauma teaching and surgical clerkship presented in this paper is felt to fulfill a need for an appropriate teaching model for use in the undergraduate surgical curriculum. Earlier work demonstrated a significant increase in factual knowledge when measured by MCQ immediately on completion of a SCORPIO trauma program.’ This controlled study dem- onstrated that a higher level of competence persisted when assessed by a performance-based examination 6 months af- ter the teaching.

REFERENCES 1. Walton HI. Proceedines of the World Summit on Medical Ed-

ucation, Med’Educ. 1993;%3(supplement 1):140-149.

THE AMERICAN JOURNAL OF SURGERY@ VOLUME 173 FEBRUARY 1997 139

2. Adunnced Trauma Life Support Course. Chicago, IL: Provider Manual 13. Baker SP. Injuries: the neglected epidemic: Stone lecture, American College of Surgeons, Committee on Trauma, 1988. 1985 American Trauma Society meeting. ] Trauma. 1987;27: 3. Lucas CE, Ledgerwood AM, Gursel E, er al. The surgical clerk- 343-348. ship and medical student education in trauma. .l Trauma. 14. Trunkey DD. Presidential address: on the nature of things rhat 1986;26:1024-1030. go bang in the night. Surgery. 1982;92:123-132. 4. Lawrence PF, Alexander RH, Bell RM, et al. Determining the 15. Stillman PL, Regan MB, Swanson DB, et al. An assessment of content of a surgical curriculum. Surgery. 1983; 94:309-3 17. the clinical skills of fourth-year students at four New England Med- 5. Snow N, Imbembo AL. Thoracic surgical content of an under- ical Schools. Acrid Med. 1990;65:320-326. graduate surgical curriculum. Surgery. 1986;100:83-88. 16. Harden RM, Sowden S, Dunn WR. Educational strategies in curric- 6. Prater MA, Smith DJ, Jr. Determining undergraduate curriculum ulum development: the SPICES model. Med Educ. 1984;18:284-297. content in plastic surgery. Plast Reconstr Surg. 1989;84:529-533. 17. Lehman RAW, Brodner RA, Greenblatt SH, et al. Clinical 7. Hill DA. SCORPIO: A system of medical teaching. Med Teach. clerkships in neurosurgery and neurology at United States Medical 1992;14:37-41. Schools. Neurosurgery. 1991;29:624-628. 8. Hill DA. A model to teach trauma care to medical students. Med 18. Duthie RR. Undergraduate orthopedic education. Med Educ. Teach. 1993;15:179-186. 1987;21:63-70. 9. Laidlaw JM, Harden RM. What is a study guide? Med Teach. 19. Maudsley RF. Effective in-training evaluation. Med Teach. 1990;12:7-12. 1989;11:285-289. 10. Hill DA, Guinea AI, McCarthy WH. Formative assessment: a 20. Newhle DI, Jaeger K. The effect of assessments and examina- student perspective. Med Educ. 1994;28:394-399. tions on the learning of medical students. Med Educ. 1983;17:165- 11. Harden RM, Stevenson M, Downie WW, Wilson GM. Assess- 171. ment of clinical competence using objective structured examina- 2 1. Biran LA. Self assessment and learnmg through GOSCE (group tion. BwvZT. 1975;1:447-451. objective structured clinical examination). Med Educ. 1991;25:475- 12. Harden RM, Gleeson FA. Assessment of clinical competence 479. using an objective structured clinical examination. Med Educ. 22. Wehber RH. Structured short-answer questions: an alternative 1979;13:41-54. examination method. Med Edtir. 1992;26:58-62.

EDITORIAL COMMENT the trauma specific educational goals, (3) a more enthused During the past generation, medical student teaching has faculty teaching in their area of their special interests, (4)

evolved due, in part, to refined teaching methods, better a more responsive student body exposed to the excitement definition of a core curriculum, and improved communi- emanating from the faculty, and (5) the tendency for im- cation techniques. Despite these advances, trauma educa- provement when any new teaching method is introduced. tion is often half-hazard reflecting its low priority in many The challenge for Hill and coworkers will be to maintain renowned medical schools. Hill and coworkers present data this enthusiasm by periodically revising the trauma core that the cognitive and psychomotor skills are higher when curriculum since all knowl’:dge is dynamic and by reinvig- students are taught within a multidiscipline trauma teach- orating the trauma faculty fzven when this requires replace- ing program designed by faculty working at a designated ment of tired, old warriors, with youthful, more articulate Level I trauma center. The significant increase in student former students. scores, when compared to their classmates taught in the Charles E. Lucas, MD traditional manner at three other hospitals, is probably mul- Department of Surgery tifactorial and includes: (1) a defined trauma core curricu- Detroit Receiving Hospital lum, (2) a trauma center coterie of faculty who designed Detroit, Michigan

140 THE AMERICAN JOURNAL OF SURGERY@ VOLUME 173 FEBRUARY 1997