Results
At the start of the study, 54 students had consented to
participate. After a detailed analysis of the participation
log, a total of 20 students had completed the study in its
entirety. The graph below (Figure 4) illustrates the overall
63% drop in participating from the start and end of the
study
Figure 4: Participation from start to end.
Using SPPS T-tests and Univariate analysis of Variance,
overall comparison of VF and A group performance
showed little difference in improvement from pre to post
test scores. Comparing all four groups, VF3 improved the
most while VF7 improved the least. However, this data
was not statistically significant. The graph below shows
the average (Avg) pre to post test improvements (learning
retention) for each of the four groups.
Figure 5: Comparison of pre to post test
improvements for (learning retention)
The graph below shows that 79.3% (n=23) of students
preferred VF over reading EBM article alone. Only 20.7%
of students did not prefer VF.
Figure 6: Shows student preference to VF over reading
alone.
Kristine Smith, Alina Kravtsova, Juan Cendan, M.D., Michael Eakins College of Medicine, University of Central Florida, March 2012
Comparison of UCF COM Virtual Family Simulation Experience or reading evidence based medicine article as a learning tool
Acknowledgments
Thank you to Alina Kravtsova (usability data), Michael
Eakins, Juan Cendan M.D., METIL, and IST for the
development and delivery of VF..
Contact email: [email protected]
References
1. Education AIfIM. Effective Use of Educational
Technology in Medical Education Colloquium on
Educational Technology: Recommendations and
Guidelines for Medical Educators. March 2007.
2. Pruthi S, Gostout BS, Lindor NM. Identification and
Management of Women with BRCA Mutations or
Hereditary Predisposition for Breast and Ovarian Cancer.
Mayo Clin Proc. Dec;85(12):1111-1120.
Aim
Objectives
The main goals of this pilot study were: 1) compare Virtual
Family to reading Evidence Based Medicine (EBM) article
as a learning tool for long term patient management, and
2) to gauge usability of Virtual Family. The results of the
study would lead us closer to the ultimate goal of
incorporating all five virtual family members developing
conditions in sync with the UCF COM curriculum.
Research Questions
1. Does the Virtual Family simulation experience
improve retention as compared with reading?
2. Do medical students prefer Virtual Family simulation
experience over reading EBM article alone?
3. Does the Virtual Family simulation experience
improve instructional condition efficiency (learning) as
compared with reading alone?
Introduction
Figure 1: Snap shot of Virtual Family
• The Virtual Family (VF) is a longitudinal educational
simulation program to be integrated into the UCF COM
four year curriculum.
• First year medical students will be responsible for the care
and management of a five member virtual patient family
who will develop several conditions over time.
• This study focuses on one case for the Mom who is at
high risk for breast cancer and needs a proper treatment
plan.
• Majority of medical schools across the country incorporate
early clinical exposure into curriculums through
preceptorship and clinical skills centers which most often
include Harvey and Standardized Patients (SPs).
• Medical students are arriving to their clerkships and
residencies more prepared and experienced for clinical
practice and learning.
Current problems: • However, these common staples of clinical exposure lead
to unavoidable disjunctive isolation.
• The fundamental question is, “…when to employ
technology during the medical education process and how
best to use it when it is employed?”1
Solution: • VF addresses this gap by providing medical students with
patients they will interact with and be responsible for throughout their four years in medical school.
Method
• Content for Virtual Family (VF) breast cancer screening
case for the Mom, “Linda Smith,” created by medical
student researchers.
• Content includes items such as: clinical vignette, patient
history, history taking, physical exam, lab ordering,
radiology, evidence based medicine article, pre-test,
post-test, within case questions and feedback.
• The web interface for VF, “Reality Medicine” developed
by UCF IST METIL.
• Evidence Based Medicine (EBM) article related to
breast cancer screening was chosen to represent
traditional reading assignment at medical schools
across the country2.
• Questions were created for pre and post tests and
came directly from this article.
• The purpose of the pre-test is to establish baseline
knowledge for each individual student, and the group
as a whole, while post-test (same questions) allows us
to compare student performances before and after the
study.
• Usability questionnaire created by IST gauges usability
and students’ overall impression of the program.
• Study delivered to first year UCF COM Class of 2015.
Group Assignments: 1. Virtual Family 3 days (VF3)-
Pre-test >EBM article >VF breast cancer screening
case >post-test/usability survey three days later.
2. Virtual Family 7 days (VF7)-
Pre- test > EBM article > VF breast cancer screening
case > post- test/ usability survey seven days later.
3. Article 3 days (A3)-
Pre- test > EBM article > post- test three days later.
Also given access to VF at the same time as VF3 and
VF7 to allow comparison of learning experiences.
4. Article 7 days (A7)-
Pre- test > EBM article > post- test seven days later.
Also given access to VF at the same time as VF3 and
VF7 to allow comparison of learning experiences.
Figure 2: Snap shot of VF breast cancer screening case
Conclusions
• Data collected is not statistically significant, it can neither
support nor undermine the hypothesis.
• As a whole VF learned (improve their test score)
approximately the same as Article group.
• Article group as a whole fared better with the decay of
time (retention) than VF.
• Content developed for breast cancer screening case will
serve as the template for future content development of
the other VF cases.
• Participation was the largest obstacle in this study.
Future studies will be of shorter duration and equal times
for all groups.
• Number of emails led to confusion, less emails and more
guidance within web interface will be used.
• Sample cases will be developed as tutorial in effort to
reduce confusion.
79.30%
20.70%
0%
20%
40%
60%
80%
100%
YES NO
Do you prefer VF over reading article alone?
Figure 3: Overview of study schedule
3 days
7 days 0%
20%
40%
VF A P
ost-
pre
tes
t A
vg
.
sc
ore
s
Learning Retention
54
20
0
10
20
30
40
50
60
Start End
# o
f P
art
icip
an
ts
Total Participation
Drop by 63%
