Page 1
2013
2013; 35: S47–S55
Using simulation to improve the cognitiveand psychomotor skills of novicestudents in advanced laparoscopic surgery:A meta-analysis
AZZAM S. AL-KADI1,2 & TYRONE DONNON2
1Qassim University, Saudi Arabia, 2University of Calgary, Canada
Abstract
Advances in simulation technologies have enhanced the ability to introduce the teaching and learning of laparoscopic surgical
skills to novice students. In this meta-analysis, a total of 18 randomized controlled studies were identified that specifically looked at
training novices in comparison with a control group as it pertains to knowledge retention, time to completion and suturing and
knotting skills. The combined random-effect sizes (ESs) showed that novice students who trained on laparoscopic simulators have
considerably developed better laparoscopic suturing and knot tying skills (d¼ 1.96, p5 0.01), conducted fewer errors (d¼ 2.13,
p5 0.01), retained more knowledge (d¼ 1.57, p5 0.01) than their respective control groups, and were significantly faster on time
to completion (d¼ 1.98, p5 0.01). As illustrated in corresponding Forest plots, the majority of the primary study outcomes
included in this meta-analysis show statistically significant support (p5 0.05) for the use of laparoscopic simulators for novice
student training on both knowledge and advanced surgical skill development (28 of 35 outcomes, 80%). The findings of this meta-
analysis support strongly the use of simulators for teaching laparoscopic surgery skills to novice students in surgical residency
programs.
Introduction
Simulators were first used in aviation for flight training of pilots
and to improve inter-staff communication (Helmreich et al.
1999; Jarm et al. 2007). The earliest documented use of
simulation for the purpose of training was for military combat
by the Roman Empire (Sokolowski & Banks 2009). In
comparison, surgery has been using the simulation method
of training for centuries in the form of animal models, cadavers
and other materials that have been used to represent tissue or
organs (Cooper & Taqueti 2004). With recent advances in
technologies, the use of high fidelity human patients, task
trainers and virtual reality simulators have enhanced teaching
and learning complex skills in surgical training programs such
as laparoscopic surgery (Satava 2008; Al-Kadi et al. 2012).
Surgical specialties are now moving forward rapidly to
incorporate simulation in medical training and education in
order to develop specific competencies and meet licensure
requirements of residents and practitioners.
William Halsted (1852–1922) at Johns Hopkins Medical
Center was instrumental in establishing the surgical residency
system into modern surgery practice (Tan & Graham 2010).
Halsted modeled his residency system from the German
system, filling it with many assistants, fewer residents and one
chief resident who held the position for two years. The chief
resident was taught by Halsted himself, and was in turn
responsible for teaching those under him (Barnes et al. 1989).
Practice points
. When used appropriately, surgical simulators have the
potential to enhance the teaching and learning oppor-
tunities for medical students and residents in advanced
laparoscopic surgery training.
. Novice students who trained on simulators for the
development of laparoscopic suturing and knot tying
skills performed at the 98th percentile in comparison
with the control groups (d¼ 1.96, p5 0.01).
. Novice students who trained on simulators were
significantly faster, performing at the 98th percentile in
comparison with the control groups (d¼ 1.98, p5 0.01).
. Novice students who trained on simulators have
conducted fewer surgical errors than the control
group, performing at the 97th percentile with a resulting
large ES difference of d¼ 2.13, p5 0.01.
. Novice students who trained on simulators retained
more knowledge than the control groups, performing at
the 73rd percentile with a resulting large ES difference of
d¼ 1.57, p5 0.01.
. Based on the findings of this meta-analysis, surgical
residency programs are highly encouraged to adopt the
use of simulators for teaching laparoscopic surgery skills
to novice students.
Correspondence: Dr Azzam S. Al-Kadi, MD MSc FRCSC, Unaizah College of Medicine, PO Box 991 Unaizah 51911, Qassim University, Qassim,
Saudi Arabia. Tel: þ966 6 361 0151; fax: þ966 6 364 9074; email: [email protected]
ISSN 0142–159X print/ISSN 1466–187X online/13/S10047–9 � 2013 Informa UK Ltd. S47DOI: 10.3109/0142159X.2013.765549
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 2
In this traditional model of teaching, residents had to
learn in the operating room through graded responsibility
under direct supervision. With a large volume of surgical
cases and time to progress from observer to primary surgeon,
this model served well for open surgical techniques but was
less optimal in learning more complex surgery techniques
such as laparoscopic surgery (Scott 2000; Hyltander et al.
2002).
In medical education context, simulation can be defined as
an education technique that allows an interactive and, at times,
immersive experience by recreating all or part of a clinical
experience without exposing patients to the associated risks
(Maran & Glavin 2003). Increasing concerns about patient
safety have focused attention on the methods used to train and
prepare doctors for clinical practice (McQuillan et al. 1998;
Perkins 2007). The use of simulation technology to enhance
skill development through deliberate practice has been shown
to be an important method to lowering surgery complications
and ultimately to reduce the risk of patient morbidity and
mortality (Issenberg et al. 1999). For example, See et al. (1993)
found that surgeons who performed clinical procedures
without additional training after their initial laparoscopy
course were 3.39 times more likely to have at least one
complication compared with surgeons who sought additional
training using simulation.
Simulation has been shown to improve the speed of novice
students and shortens the overall time needed for them to
complete certain laparoscopic procedures (Korndorffer et al.
2005 (a) and (b); Aggarwal et al. 2007). Furthermore, a
complex psychomotor skill that needs more dedicated training
and expected to be mastered by more senior trainees like
laparoscopic suturing has also been shown to be improved
with simulation (Fried et al. 2004).
The purpose of this meta-analysis was to determine the
effectiveness of using simulation to enhance the knowledge
and skill competencies of novice students in advanced
laparoscopic surgery; specifically as it pertains to knowledge
retention, time to completion, surgical errors and suturing and
knotting.
Methods
An electronic search was performed for all peer-reviewed
studies focusing on the use of simulation for training novice
students laparoscopic surgery skills from January 1999 to 2012.
In addition to Google Scholar and the National Library of
Medicines’ PubMed, we searched the Education Resources
Information Center (ERIC) and psychological databases
(PsychInfo, Washington, DC). The reference lists of the initial
primary studies identified from this search were also examined
to locate other potential studies to be included in this meta-
analysis.
The following search terms were used to identify the
studies: ‘‘simulator’’, ‘‘laparoscopic surgery’’, ‘‘surgical train-
ing’’, ‘‘novice’’ and ‘‘meta-analysis’’ to retrieve a total of 559
articles (Figure 1). Only randomized controlled trials (RCT)
published in English-language journals that assessed the
effectiveness of simulator training on knowledge retention,
time to completion and suturing and knotting compared with
video training, no training or a standard laparoscopic training
procedure were included.
Out of the 559 articles collected initially, 521 articles were
excluded because they did not meet the necessary criteria for
inclusion (e.g. not a peer-reviewed journal publication, a non-
randomized study, or without a control group). Full copies of
the remaining 38 studies were retrieved and the two authors
independently critiqued the articles based on the pre-
established inclusion and exclusion criteria. A manual search
of the references identified eight more potential studies.
Moreover, a final review yielded a total number of 18 eligible
studies to be included in the meta-analysis.
The exclusion of articles during the final reviewing process
was primarily due to enrolling advanced-trained surgeons as
participants instead of novice in the intervention or control
group, and more importantly studies that did not provide
sufficient statistical information (e.g. means and standard
deviations, F ratio or t test statistics) needed to calculate the
overall ES differences between groups.
Inclusion and exclusion criteria for eligible studies
The inclusion and exclusion criteria were formulated based on
a comprehensive review of the literature and published articles
in the field of surgical simulators and simulators in laparo-
scopic surgery training in particular. To be included in
Figure 1. Flow chart for the selection and reviewing process
in this meta-analysis.
A. S. Al-Kadi & T. Donnon
S48
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 3
accordance with the purpose of this meta-analysis, a study has
to meet the following eligibility criteria:
(1) Published peer-reviewed Randomized Control Trials in
English language only with the date of publication from
January 1999 to January 2012.
(2) Eligible studies have to contain sufficient statistical
information (e.g. means and standard deviations, F and
t test statistics).
(3) Participants in both groups must be:
(a) Novice laparoscopic surgery students (e.g. med-
ical students).
(b) Or trainees with limited laparoscopic experience
(we defined limited as less than 25 laparoscopic
procedures in the past).
(4) Types of intervention using surgical simulators:
(a) Comparing two groups: a control group and an
intervention group who underwent simulation
training exposure.
(b) Scoring done by assessing students using consis-
tent measures between groups on pre- and post-
test assessments.
(5) The included studies must have contained information
on measures of the following outcomes:
(a) Laparoscopic suturing and knot tying scores
(b) Time to complete the assigned task
(c) Error score
(d) Retaining knowledge of instruments and
procedures.
Coding protocol and data extraction
The coding protocol for extracting data was developed based
on careful review of the related literature. All outcome
measures were investigated in relation to the possible
independent variables that may have had an effect and
influence on the dependent variables (e.g. time to
complete task).
The coding protocol developed for this meta-analysis
included the study’s title, author’s name(s), year, source of
publication, study design, simulation type, surgical proce-
dure performed, sample size, assessment method and
measured outcomes. As one of the criteria was that the
study had to be a RCT, the quality of the studies research
designs was consistent. All 18 articles that met the inclusion
and exclusion criteria were later coded independently and
reviewed by the authors until 100% agreement was
obtained.
Statistical analysis
We used the Comprehensive Meta-Analysis software program
(version 1.0.23, Biostat Inc., Englewood, NJ) for statistical
analysis. Both reviewers checked that all data extracted was
entered accurately and the analysis was completed correctly.
The ES and 95% confidence intervals for each study outcome
was calculated using Cohen’s d standard formula for the mean
difference between two groups:
d ¼MT �MC=�pooled and �pooled
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi�2
1 þ �22
� �=2
� �qwhere MT and MC are the means for the treatment (T) and
control (C) groups, respectively, and �1 and �2 are the
standard deviations for the treatment and control groups,
respectively (Cohen 1988).
When the means and standard deviations were not
reported, we were able to calculated the ES from other
reported statistical analyses such as the F ratio or t test results
using the following formulas:
ESsm ¼
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiF n1 þ n2ð Þ
n1n2
sor ESsm ¼ t
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffin1 þ n2ð Þ
n1n2
s,
where ESsm is the ES for the standardized mean difference,
n1 is the control group sample size, and n2 is the test group
sample size (Cohen 1988).
We employed a random-ES model (DerSimonian and
Laird) in combining the weighted and unweighted ESs
because such model reflects a conservative estimate of the
between-study variance and the overall psychometric skill
score. To investigate the heterogeneity of studies as well as
identify outliers or naturally occurring independent variable
groupings that could be explained by examining modera-
tors, we plotted the standard deviations of the unweighted
ESs. In addition, the combined ESs generated in the Forrest
plots were examined for heterogeneity using Cochrane Q
tests and were considered significant if p values
were 50.05.
The combined ESs were calculated irrespective of the
number of trials included under each outcome in order to
obtain a uniform ES estimate based on the domain of
measurement (i.e. laparoscopic suturing). If the calculated ES
(Cohen’s d) is 0.30–0.49, then it is considered generally a
‘‘small’’ ES difference. ES of d¼ 0.50–0.79 is interpretted as a
‘‘medium’’ effect, and anything equal to d¼ 0.80 or greater is
considered a ‘‘large’’ ES difference. Significant probability
values was set at p5 0.05.
Results
As shown in Table 1, there are a total of 18 studies included in
this meta-analysis with a combined number of n¼ 451 student
participants. In all the studies included in our meta-analysis, all
participants underwent an assessment of their baseline skills
before the educational intervention and both groups (treat-
ment and control) were reported to be equal in their baseline
characteristics (i.e. knowledge and skill acquisition). The
assessment measures used to assess their baseline perfor-
mance knowledge and skill competencies were the same used
to evaluate the trainees at their post-test assessments. This
means that moderators like the novices’ level of laparoscopic
experience and baseline laparoscopic knowledge, which have
the potential to alter the magnitude of treatment effect, are
controlled before the simulation intervention was
implemented.
Laparoscopic surgery simulation
S49
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 4
Laparoscopic suturing and knot tying skills
In the seven studies that reported outcomes on laparoscopic
suturing and knot tying skills, combined estimates using the
conservative random effects size calculation showed that
students who were trained on simulator have a higher quality
laparoscopic suturing and knot tying performance scores (i.e.
faster, more efficient and placed well-secured knots). As
shown in Figure 2, in 12 of the 15 (80%) outcomes measures
extracted the group of students who received training using
the simulator scored significantly higher on their post-test
assessments. The combined ES was large using the random ES
model d¼ 1.96 (95% CI: 0.91–3.01) and statistically significant
(p5 0.01). Test for heterogeneity showed a Cochran Q values
of 289.10 using Cohen’s standard and 69.80 with Hodges’s
standard. Both with degrees of freedom of 14, p values
50.001.
Time to complete the assigned task
In the nine studies that reported outcomes on how fast novice
students completed a certain laparoscopic task, combined
estimates using the random-effects size calculation model
showed that novice students who trained on simulators were
significantly faster when performing different assigned tasks
and spent shorter overall time to completion. As shown in
Figure 3, in 12 of the 16 (75%) outcomes measures extracted
the group of students who received laparoscopic training using
the simulator scored significantly higher on their post-test
assessments (p5 0.05). The combined effect using the
random effects size model was found to be large, d¼ 1.98
(95% CI: 1.28–2.68) and statistically significant (p5 0.01).
Test for the heterogeneity showed a Cochran Q values of
621.71 using Cohen’s standard and 173.45 with Hodges’s
standard. Both with degrees of freedom of 27, p values
50.001.
Error scores
Out of the eight studies that reported outcomes on the number
of errors conducted by students while performing different
laparoscopic tasks, meta-analytic estimates with the conserva-
tive random-ES calculation showed that novice students who
trained on simulators conducted fewer errors than the control
group. As shown in Figure 4, the group of students who
received the training using the simulator scored higher in their
post-test assessment. The combined effect using the random
ES model was large at d¼ 2.13 (95% CI: 1.28–2.97) and
statistically significant (p5 0.01). Test for the heterogeneity
showed a Cochran Q values of 145.39 using Cohen’s standard
and 38.00 with Hodges’s standard. Both with degrees of
freedom of 13, p values 50.001.
Retaining knowledge of instruments and procedures
Out of the two studies that reported outcomes on retaining
knowledge of the procedure and instruments, combined
estimates using the random effects size model showed that
novice students who trained on simulators retained more
knowledge than the control group. As shown in Figure 5, in all
four of the outcomes measures extracted from the two studies,
Table 1. List of the 18 studies included in the meta-analysis.
References Total no. Type of simulator Type of procedure
1 Ahlberg et al. (2007) 13 LapSim Laparoscopic tubal ligation
2 Aggarwal et al. (2007) 20 LapSim Laparoscopic cholecystectomy
3 Andreatta et al. (2006) 19 LapMentor Camera navigation skills, efficiency of motion,
instrument handling, perceptual ability, safe
electrocautery, safe clipping
4 Clevin et al. (2008) 16 LapSim Camera navigation, instrument navigation,
coordination, grasping, lifting and grasping,
cutting, clip applying
5 Fried et al. (2004) 20
12
MISTELS
MISTELS
Intracorporal and extracorporal knot tying
Intracorporal and extracorporal knot tying
6 Ganai et al. (2007) 19 Angled-telescope simulator Laparoscopic camera navigation
7 Grantcharov et al. (2004) 20 MIST-VR Laparoscopic cholecystectomy
8 Hyltander et al. (2002) 24 LapSim Instrument navigation, camera navigation and
coordination
9 Jordan et al. (2001) 32 U/Z maze boxes &MIST VR Laparoscopic cutting skill
10 Korndorffer Jr et al. (2005) 17 VT Intracorporal knot tying
11 Lucas et al. (2008) 32 LapMentor Laparoscopic cholecystectomy
12 Madan et al. (2007) 65 MIST-VR & LTS 2000 Placing a piece of bowel in retrieval bag,
performing a liver biopsy, placing a stapler
on the bowel, ‘‘running’’ the bowel
13 Scott et al. (2000) 27 VT (Karl Storz endoscopy) Laparoscopic cholecystectomy
14 Seymour et al. (2002) 16 MIST-VR Laparoscopic cholecystectomy
15 Stefanidis et al. (2007) 32 VT Intracorporal knot tying
16 Stefanidis et al. (2008) 15 VT Intracorporal knot tying
17 Van Sickle et al. (2008) 22 MIST-VR Intracorporal knot tying
18 Verdaasdonk et al. (2008) 20 VR simulator (SIMENDO, DelltaTech) Intracorporal knot tying
Notes: MIST-VR¼minimally invasive surgical trainer-virtual reality, LapSim¼ laparoscopic simulator, MISTELS¼McGill inanimate system for training and evaluation of
laparoscopic skills, LapMentor¼ laparoscopic mentor simulator, LTS¼ Laparoscopy Training Simulator 2000, VR¼ virtual reality, VT¼ video trainer.
A. S. Al-Kadi & T. Donnon
S50
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 5
the group of students that received the laparoscopic
training using the simulator scored significantly higher on
their post-test knowledge assessments (p5 0.01). The com-
bined effect was large using the random effects size model,
d¼ 1.57 (95% CI: 1.06–2.07) and statistically significant;
p5 0.01. Test for the heterogeneity showed a Cochran Q
values of 5.11 using Cohen’s standard and 2.80 with Hodges’s
standard. Both with degrees of freedom of 3, p values 40.05.
Discussion
When used appropriately, surgical simulators have the
potential to enhance the teaching and learning opportunities
for medical students and residents in laparoscopic surgery
training. As shown in this meta-analysis, large combined
random-ESs were reported in randomized controlled studies
for novice students as it pertains to (1) knowledge retention
[d¼ 1.57 (95% CI: 1.06–2.07), p5 0.01] and (2) advanced skill
development as a function of: (a) suturing and knot tying
[d¼ 1.96 (95% CI: 0.91–3.01), p5 0.01], (b) time to
completion [d¼ 1.98 (95% CI: 1.28–2.68), p5 0.01] and (c)
error score [d¼ 2.13 (95% CI: 1.28–2.97), p5 0.01]. Another
way to interpret these results is to compare the average
percentile standing of the intervention (experimental) group
relative to the control group based on the combined ES value.
Large ESs of d¼ 1.50 and d¼ 2.00 respectively indicate that the
mean of the intervention group is at the 93.3th and 97.7th
percentile of the control group at post-assessment perfor-
mance levels.
Meta-analysis of laparoscopic suturing and knottying scores
Most laparoscopic surgeons believe that laparoscopic skills
vary in difficulty level. Clipping the cystic artery, for example,
is a task that every surgical resident should be comfortable
performing during a laparoscopic cholecystectomy. He or she
is expected to master such skill before the completion of
residency training. In contrast, laparoscopic intracorporal knot
tying is a more complex skill which is expected to be mastered
by only fellows and senior trainees.
In the seven primary studies included in our meta-analysis
that reported outcomes on laparoscopic suturing and knot
tying skills, combined ES estimates showed that novice
students who trained on simulators performed at the 98th
percentile in comparison with the control groups (d¼ 1.96,
p5 0.01). Therefore, the simulator as a teaching and learning
tool has created a noticeable difference among novice students
not only in their ability to successfully complete the basic
laparoscopic skills, but also in achieving more advanced skills
like intracorporal knot tying. The primary studies were also
found to p5 0.001.
Citation Effect name N Total Effect Lower Upper P value -8.0 (worse) 0 +8.0 (better)
Ahlberg et al. 2007 Loop ligation score 29 0.25 -0.52 1.02 0.51
Fried et al. 2004 IC knot tying score 20 1.24 0.21 2.27 0.01
Fried et al. 2004 EC knot tying score 20 -0.43 -1.38 0.52 0.35
Korndorffer Jr et al. 2005 Suturing Speed 17 1.52 0.34 2.70 0.01
Korndorffer Jr et al. 2005 Suturing Score 17 1.50 0.33 2.67 0.01
Korndorffer Jr et al. 2005 Knot Security 17 0.39 -0.66 1.44 0.43
Scott et al. 2000 TC (running string) 22 5.80 3.76 7.84 0.00
Scott et al. 2000 Foam suturing speed 22 6.96 4.59 9.33 0.00
Stefanidis et al. 2007 Suturing score (tough) 19 2.22 0.93 3.51 0.00
Stefanidis et al. 2007 Suturing score (easy) 19 2.14 0.87 3.41 0.00
Stefanidis et al. 2007 Suturing/Tying score 15 2.70 1.05 4.34 0.00
Van Sickle et al. 2008 Suturing speed 22 1.45 0.45 2.45 0.00
Van Sickle et al. 2008 Needle manipulation 22 0.92 -0.02 1.86 0.04
Verdaasdonk et al. 2008 Knot tying speed 20 1.13 0.11 2.14 0.02
IC = intracorporal, EC = extracorporal, TC = time to complete.
Verdaasdonk et al. 2008 Driving a needle speed 20 1.65 0.56 2.74 0.00
Fixed-Effect Model 301 1.94 1.71 2.17 0.00
Random-Effects Model 301 1.96 0.91 3.01 0.00
Figure 2. Random and fixed ES models for the combined effect of ‘laparoscopic suturing and knot tying skill’ scores.
Laparoscopic surgery simulation
S51
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 6
Meta-analysis of time to complete an assignedtask scores
One of the major drawbacks of laparoscopy is the extra time
needed thereby prolonging the overall surgical procedure
compared to the same procedure performed in an open
fashion. This concern has been resolved to some extent with
the progress made in training novices on some basic
laparoscopic skills before scrubbing in the OR. The total time
for performing a certain procedure gets shorter as residents
master various laparoscopic skills with deliberate practice
beforehand but it will hardly be at the same level as those of
the supervisor staff. Because residents are required to perform
a part, or all, of the procedure depending on the level of
training, this will create a delay and lengthen surgery time. This
delay can impede the turnover between surgical cases and
slow down the overall flow of straightforward operations.
In the nine primary studies included in our meta-analysis
that reported outcomes on the students’ time to completion
while performing different laparoscopic tasks, combined ES
estimates shows that novice students who trained on
simulators were significantly faster, performing at the 98th
percentile, in comparison with the control groups (d¼ 1.98,
p5 0.01). This finding suggests that the use of simulation
enhances the speed of novice students dramatically and gives
them the ability to consume less time while performing
different laparoscopic procedures.
Time to complete the laparoscopic procedure and subse-
quently the speed of performing a certain laparoscopic tasks
have been major concerns to surgeons since the introduction
of laparoscopic technology (Agachan et al. 1997). However, as
students progress with each completed task towards mastering
laparoscopic skills, this issue becomes less prominant. Our
meta-analysis supports the learning curve theory for novice
students where their total time required to accomplish a
laparoscopic task significantly decreases with more practice
using simulators.
Meta-analysis of error scores
While most medical errors are the team’s fault, errors that
occur during a surgical or interventional procedure are related
to surgical performance outcomes which are basically the fault
of the surgeon (Satava 2005). An expected outcome from using
simulations is to minimize the morbidity and mortality of
patients even though it has not been directly connected to
measured patient outcomes.
In the eight primary studies that reported outcomes on
error scores, meta-analytic estimates shows that novice
Citation Effect name N total Effect Lower Upper P value -8.0 (slow) 0 +8.0 (fast)
Aggarwal et al. 2007 TC (LC) 19 5.53 3.40 7.66 0.00
Andreatta et al. 2006 TC (needle transfer) 19 1.57 0.46 2.68 0.00
Andreatta et al. 2006 TC (cam. nav.) 19 1.17 0.12 2.21 0.02
Clevin et al. 2008 TC (multiple tasks) 16 1.46 0.25 2.67 0.01
Ganai et al. 2007 TC (navigation) 19 1.15 0.10 2.20 0.02
Grantcharov et al. 2004 TC (LC) 16 1.07 -0.08 2.22 0.05
Hyltander et al. 2002 TC (all tasks) 24 5.50 3.64 7.34 0.00
Korndorffer Jr et al. 2005 TC (cam. nav.) 20 0.16 -0.80 1.10 0.73
Madan et al. 2007 TC (placing bowel in bag) 34 0.71 -0.01 1.43 0.05
Madan et al. 2007 TC (running bowel) 34 0.90 0.17 1.64 0.01
Madan et al. 2007 TC (stapling bowel) 34 0.34 -0.36 1.05 0.32
Madan et al. 2007 TC (liver Bx) 34 0.80 0.73 2.86 0.00
Scott et al. 2000 TC (checkerboard) 22 1.80 0.73 2.87 0.00
Scott et al. 2000 TC (running bowel) 22 5.80 3.76 7.84 0.00
TC = time to complete, LC = Laparoscopic Cholecystectomy.
Scott et al. 2000 TC (bean drop) 22 4.00 2.45 5.55 0.00
Scott et al. 2000 TC (block move) 22 2.82 1.55 4.09 0.00
Fixed-Effect Model 376 1.54 1.82 2.23 0.00
Random-Effects Model 376 2.17 1.22 3.12 0.00
Figure 3. Random and fixed ES models for the combined effect of ‘‘time to complete’’ scores.
A. S. Al-Kadi & T. Donnon
S52
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 7
students who trained on simulators conducted fewer errors
and were performing at the 97th percentile in comparison with
the control group (d¼ 2.13, p5 0.01).
Traditional educational processes have focused on
‘‘teaching the correct thing’’ to the extent that there is no
model for teaching that allows for working through errors. In
making errors, a student can learn to recognize when an error is
about to occur and avoid it and likewise, to correct an error that
occurred before a complication can develop (Satava 2008).
Meta-analysis of knowledge retention scores
While simulators have proven to be very effective in
psychomotor skills training, their benefits can go beyond that
to involve cognitive skills as well. In the two primary studies
that reported outcomes on knowledge retention scores
between novice students who trained on simulator and control
group, combined ES estimates shows that novice students who
trained on simulators retained more knowledge than the
control groups, performing at the 73rd percentile, with a
Citation Effect name N Total Effect Lower Upper P value -8.0 (worse) 0 +8.0 (better)
Ahlberg Error rate 13 5.78 3.00 8.56 0.00
Clevin BV injury error 16 1.07 -0.08 2.22 0.05
Ganai TE (navigation) 19 2.46 1.18 3.74 0.00
Ganai Scope smudge error 19 2.77 1.41 4.13 0.00
Ganai Instr. Collision error 19 1.12 0.08 2.16 0.03
Ganai Horizone error score 19 1.33 0.26 2.40 0.01
Grantcharov Error rate (LC) 16 1.46 0.25 2.67 0.01
Jordan Foam suturing speed 16 0.91 -0.21 2.04 0.09
Jordan Suturing score (tough) 16 0.11 -0.97 1.18 0.84
Jordan Suturing score (easy) 16 3.10 1.51 4.70 0.00
Seymour Suturing/Tying score 16 5.70 3.29 8.11 0.00
Seymour Suturing speed 16 1.48 0.27 2.69 0.01
Van Sickle et al. 2008 TE (suturing) 22 1.18 0.22 2.14 0.01
Verdaasdonk et al. 2008 Error score (knot tying) 20 1.43 0.38 2.48 0.00
BV= blood vessels, TE= total error, Instr= instrument, LC = Laparoscopic Cholecystectomy.
Fixed-Effect Model 243 2.04 1.79 2.30 0.00
Random-Effects Model 243 2.13 1.28 2.97 0.00
Figure 4. Random and fixed ES models for the combined effect of ‘‘error’’ scores.
Proced= procedure, LC = Laparoscopic Cholecystectomy , Instr= instrument.
Citation Effect name N Total Effect Lower Upper P value -8.0 (Less) 0 +8.0 (More)
Lucas et al. 2008 Knowledge of Proced. (LC) 32 1.20 0.42 1.98 0.00
Lucas et al. 2008 Knowledge of Instr. (LC) 32 1.39 0.59 2.19 0.00
Scott et al. 2000 Knowledge of Proced. (LC) 22 2.40 1.22 3.58 0.00
Scott et al. 2000 Knowledge of Instr. (LC) 22 1.42 0.41 2.43 0.00
Fixed-Effect Model 108 1.54 1.16 1.92 0.00
Random-Effects Model 108 1.57 1.06 2.07 0.00
Figure 5. Random and fixed ES models for the combined effect of ‘‘retaining knowledge’’ scores.
Laparoscopic surgery simulation
S53
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 8
resulting large ES difference of d¼ 1.57, p5 0.01. This result
indicates that novices who trained on simulators were more
able to internalize the theoretical knowledge including
indications, contraindications, steps of the procedure and
the name and function of instruments used in the
laparoscopic tasks.
In conclusion, the combined ESs were large for all the
learning outcomes that we intended to assess and address as a
rational for our analysis. The impact of using simulators in
improving the learners’ psychomotor performance in
advanced laparoscopic surgery skills was apparent from the
results of this meta-analysis. Similarly, our results have pointed
out that simulation is effective in enhancing the cognitive skills
and the knowledge of procedures/instruments by novices.
Skills that are linked and relevant to patients’ safety were also
improved with simulation as shown by our meta-analysis
Conclusion
The lack of relevant, realistic, hands-on educational experi-
ences for medical professionals may contribute to reports
indicating that between 44 000 and 98 000 deaths per year are
attributable to preventable medical errors (Kahn 1995; Berwick
& Leape 1999). The Canadian Medical Association reports that
24 000 people die each year due to medical and surgical errors
and more than 87 000 patients in Canada experience an
adverse event (Baker et al. 2004). The results of this meta-
analysis indicates (although not directly measured by patient’s
morbidity or mortality) that skills linked and related to patients’
safety were also improved (i.e. minimizing errors). Students
provided with training using simulated laparoscopic tasks
were able to internalize and recall the relevant information and
skills required more successfully after training on simulation
than control groups. Based on the findings of this meta-
analysis, surgical residency programs are highly encouraged to
adopt the use of simulators in teaching advanced laparoscopic
surgery skills to novice students.
One of the most important benefits of using surgical
simulators is that it gives students permission to take the time
to explore and practice tasks repeatedly in a nonthreatening
environment that allows for the opportunity to learn from
one’s mistakes (Stava 2008). Surgical simulators give the
opportunity for independent learning that is based on
constructive feedback received from well-trained instructors
to develop good surgical habits. In general, surgical simulators
have the remarkable attributes of being consistent (standar-
dized), repeatable, quantifiable, precise, always available, and
most importantly, objective (Nackman et al. 2003; Szekely
2003; Uranus et al. 2004; Satava 2005; Sorensen et al. 2006).
The publication of this supplement has been made possible
with the generous financial support of the Dr Hamza Alkholi
Chair for Developing Medical Education in KSA.
Declaration of interest: The authors report no declarations
of interest.
Notes on contributors
Azzam S. Al-Kadi, MD, MSc, FRCSC, is an assistant professor at the
Department of Surgery, Unaizah College of Medicine, Qassim University,
Qassim, Saudi Arabia.
Tyrone Donnon is an associate professor at the Medical Education and
Research Unit, Department of Community Health Sciences, Faculty of
Medicine, University of Calgary, Calgary, Canada.
References
Agachan F, Joo JS, Sher M, Weiss EG, Nogueras JJ, Wesner SD. 1997.
Laparoscopic colorectal surgery. Do we get faster? Surg Endosc
11:331–335.
Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A. 2007.
Proving the effectiveness of virtual reality simulation for training in
laparoscopic surgery. Ann Surg 246:771–779.
Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky
3rd DA, Ramel S, Smith CD, Arvidsson D. 2007. Proficiency-based
virtual reality training significantly reduces the error rate for residents
during their first 10 laparoscopic cholecystectomies. Am J Surg
193:797–804.
Al-Kadi AS, Donnon T, Oddone Paolucci E, Mitchell P, Debru E, Church N.
2012 Nov. The effect of simulation in improving students’ performance
in laparoscopic surgery: A meta-analysis. Surg Endosc 26(11):3215–24,
PubMed PMID: 22648101.
Andreatta PB, Woodrum DT, Birkmeyer JD, Yellamanchilli RK, Doherty
DM, Gauger PG, Minter RM. 2006. Laparoscopic skills are improved
with LapMentor training: Results of a randomized, double-blinded
study. Ann Surg 243:854–860.
Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, Etchells E, Ghali
W, Hebert P, Majumdar SR, et al. 2004. The Canadian Adverse Events
Study: The incidence of adverse events among hospital patients in
Canada. CMAJ 170:1678–1686.
Barnes RW, Lang NP, Whiteside MF. 1989. Halstedian technique revisited.
Innovations in teaching surgical skills. Ann Surg 210:118–121.
Berwick DM, Leape LL. 1999. Reducing errors in medicine. Qual Health
Care 8:145–146.
Clevin L, Grantcharov TP. 2008. Does box model training improve surgical
dexterity and economy of movement during virtual reality laparoscopy?
A randomised trial. Acta Obstet Gynecol Scand 87:99–103.
Cohen J. 1988. Statistical power analysis for the behavioral sciences.
2nd ed. Hillsdale, NJ: L Erlbaum Associates.
Cooper JB, Taqueti VR. 2004. A brief history of the development of
mannequin simulators for clinical education and training. Qual Saf
Health Care 13:si11–si18.
Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu
G, Andrew CG. 2004. Proving the value of simulation in laparoscopic
surgery. Ann Surg 240:518–525.
Ganai S, Donroe JA, St Louis MR, Lewis GM, Seymour NE. 2007. Virtual-
reality training improves angled telescope skills in novice laparosco-
pists. Am J Surg 193:260–265.
Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-
Jensen P. 2004. Randomized clinical trial of virtual reality simulation for
laparoscopic skills training. Br J Surg 91:146–150.
Helmreich RL, Merrit AC, Wilhem JA. 1999. The evolution of Crew Resource
Management training in commercial aviation. Int J Aviat Psychol
9:19–32.
Hyltander A, Liljegren E, Rhodin PH, Lonroth H. 2002. The transfer of basic
skills learned in a laparoscopic simulator to the operating room. Surg
Endosc 16:1324–1328.
Issenberg SB, McGaghie WC, Hart IR, Mayer JW, Felner JM, Petrusa ER,
Waugh RA, Brown DD, Safford RR, Gessner IH, et al. 1999. Simulation
technology for health care professional skills training and assessment.
JAMA 282:861–866.
Jarm T, Kramar P, Zupanic A. (eds). 2007. 11th Mediterranean
conference on medical and biological engineering and computing,
held in Ljubljana, Slovenia, June 2007, Vol. 16, pp. 327–328.
A. S. Al-Kadi & T. Donnon
S54
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
Page 9
Jordan JA, Gallagher AG, McGuigan J, McClure N. 2001. Virtual reality
training leads to faster adaptation to the novel psychomotor restrictions
encountered by laparoscopic surgeons. Surg Endosc 15:1080–1084.
Kahn KL. 1995. Above all ‘do no harm’. How shall we avoid errors in
medicine? JAMA 274:75–76.
Korndorffer Jr JR, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ.
2005a. Simulator training for laparoscopic suturing using
performance goals translates to the operating room. J Am Coll Surg
201:23–29.
Korndorffer Jr JR, Hayes DJ, Dunne JB, Sierra R, Touchard CL, Markert RJ,
Scott DJ. 2005b. Development and transferability of a cost-
effective laparoscopic camera navigation simulator. Surg Endosc
19:161–167.
Lucas S, Tuncel A, Bensalah K, Zeltser I, Jenkins A, Pearle M, Cadeddu J.
2008. Virtual reality training improves simulated laparoscopic surgery
performance in laparoscopy naive medical students. J Endourol
22:1047–1051.
Madan AK, Frantzides CT. 2007. Prospective randomized controlled trial of
laparoscopic trainers for basic laparoscopic skills acquisition. Surg
Endosc 21:209–213.
Maran NJ, Glavin RJ. 2003. Low- to high-fidelity simulation – a continuum of
medical education? Med Educ 37:S22–S28.
McQuillan P, Pilkington S, Allan A, Taylor B, Short A, Morgan G, Nielsen M,
Barrett D, Smith G. 1998. Confidential inquiry into quality of care before
admission to intensive care. BMJ 316:1853–1858.
Nackman GB, Bermann M, Hammond J. 2003. Effective use of human
simulators in surgical education. J Surg Res 115:214–218.
Perkins GD. 2007. Simulation in resuscitation training. Resuscitation
73:202–211.
Satava RM. 2005. Identification and reduction of surgical error using
simulation. Minim Invasive Ther Allied Technol 14:257–261.
Satava RM. 2008. Historical review of surgical simulation- a personal
perspective. World J Surg 32:141–148.
Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfav ST, Valentine RJ, Euhus
DM, Jeyarajah DR, Thompson WM, Jones DB. 2000. Laparoscopic
training on bench models: Better and more cost effective than operating
room experience? .J Am Coll Surg 191:272–283.
See WA, Cooper CS, Fisher RJ. 1993. Predictors of laparoscopic
complications after formal training in laparoscopic surgery. JAMA
270:2689–2692.
Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen
DK, Satava RM. 2002. Virtual reality training improves operating room
performance: Results of a randomized, double-blinded study. Ann Surg
236:458–463.
Sokolowski JA, Banks CM. 2009. Principles of modeling and simulation: A
multidisciplinary approach. Hoboken, NJ: John Wiley & Sons.
Sorensen TS, Greil GF, Hansen OK, Mosegaard J. 2006. Surgical simulation –
A new tool to evaluate surgical incisions in congenital heart disease?
.Interact Cardiovasc Thorac Surg 5:536–539.
Stefanidis D, Acker C, Heniford BT. 2008. Proficiency-based laparoscopic
simulator training leads to improved operating room skill that is
resistant to decay. Surg Innov 15:69–73.
Stefanidis D, Korndorffer Jr JR, Markley S, Sierra R, Heniford BT, Scott DJ.
2007. Closing the gap in operative performance between novices and
experts: Does harder mean better for laparoscopic simulator training?
.J Am Coll Surg 205:307–313.
Szekely G. 2003. Surgical simulators. Minim Invasive Ther Allied Technol
12:14–18.
Tan SY, Graham C. 2010. Medicine in stamps. Ivan Petrovich Pavlov
(1849–1936): Conditioned reflexes. Singapore Med J 51:1–2.
Uranus S, Yanik M, Bretthauer G. 2004. Virtual reality in laparoscopic
surgery. Stud Health Technol Inform 104:151–155.
Van Sickle KR, Ritter EM, Baghai M, Goldenberg AE, Huang IP, Gallagher
AG, Smith CD. 2008. Prospective, randomized, double-blind trial of
curriculum-based training for intracorporeal suturing and knot tying.
J Am Coll Surg 207:560–568.
Verdaasdonk EG, Dankelman J, Lange JF, Stassen LP. 2008. Transfer validity
of laparoscopic knot-tying training on a VR simulator to a realistic
environment: A randomized controlled trial. Surg Endosc 22:1636–1642.
Laparoscopic surgery simulation
S55
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y 93
.180
.53.
211
on 0
2/20
/14
For
pers
onal
use
onl
y.
