40
The NCSBN National Simulation Study: Results from Year 1 2012 NCSBN Scientific Symposium September 11, 2012 Arlington, VA
The NCSBN National Simulation Study: Results from Year 1
2012 NCSBN Scientific Symposium
September 11, 2012
Arlington, VA
Study Collaborators
Jennifer Hayden, MSN, RN NCSBN, Project Director
Pamela Jeffries, PhD, RN, FAAN, ANEFJohns Hopkins University
Suzan Kardong-Edgren, PhD, RN, ANEFBoise State University
Washington State University
Nancy Spector, PhD, RNNCSBN
Study Phases
Phase II-Multi-site, randomized study of simulation/clinical experiences
Purpose
Determine if there are differences in clinical competency among graduating nursing students with 50%, 25% or up to 10% of traditional clinical hours substituted with simulation experiences.
Study Groups
Control Group (clinical as usual) Up to 10% of clinical time can be in simulation
25% simulation in place of clinical 50% simulation in place of clinical
Research Questions
1. Are there differences in clinical competency among graduating nursing students in the three study groups?
2. Are there differences in knowledge among graduating nursing students in the three study groups?
3. Are their perceived differences in how well learning needs are met in the clinical and simulation environments among the three study groups?
4. Are there differences in clinical competency among the three study groups in each of the core clinical courses?
Null Hypothesis
There are no differences between the study groups in knowledge, clinical competency, and learning needs being met.
Substitution Rate
1 hour of traditional clinical is being replaced with 1 hour of simulation time
Time spent in a simulation lab either as an active participant or active observer of a simulated scenario followed by debriefing
This time in the simulation lab is counted towards required clinical hours for graduation
Core Clinical Courses
Randomization will be maintained in the core clinical courses
Foundations Medical/Surgical Nursing Advanced Medical/Surgical Nursing Obstetrical Nursing Pediatric Nursing Mental Health/Psychiatric Nursing Community Health/Public Health Nursing
Study Will Evaluate:
Clinical competencyCreighton Competency Evaluation Instrument
(CCEI)
Nursing knowledgeATI Comprehensive Assessment and Review
Program
How well student learning needs were met in the clinical and simulation environmentsClinical Learning Environment Comparison Survey
(CLECS)
Defining Simulation
Operational definition: An activity or event replicating clinical practice. For the purpose of this study, simulation includes multiple types of scenario based simulation using high fidelity manikins, medium fidelity manikins, standardized patients, role playing, and computer based simulations.
NLN/Jeffries Simulation Framework
Study Management at each School
Team Leader Study Team
Trained in the protocol Trained in the debriefing technique Large part in the development of the simulation
curriculum
Clinical Faculty Training
Clinical faculty rate students each week of clinical and each day of simulation using the Creighton Competency Evaluation Instrument
Each semester clinical faculty undergo online training in the use of the instrument
Calculate inter-rater reliability and intra-rater reliability each semester
Clinical faculty review objectives and “norm” what are competent/not competent expectations
Simulation Day
Entire clinical group reports to the lab with the instructor
Assigned roles: primary nurse, secondary nurse, family member, documenter, med nurse, observers
15-20 minutes for scenario followed by 20+ minutes debriefing
Clinical instructors rate their students with the CCEI
2-3 clinical groups in the lab at the same time
Stations
Manikin scenarios Standardized patients/patient actors Skills Stations Group Activities
Layered Learning Critical Thinking Cards Sensory kits Group Therapy in a Box
Computerized scenarios ATI Real Life Virtual Clinical Excursions
Scenarios Available
CourseNumber of Scenarios
Available
Foundations 26
Medical-Surgical 64
Advanced Medical-Surgical 46
Obstetrical 42
Pediatrics 67
Psychiatric-Mental Health 24
Community Health 23
Total 292
Student “Safety”
Safety data collected each day of simulation and each week of clinical
Oversight Committee at each school Reviews school level safety data, mid-term
grades, final grades, ATI scores, feedback from faculty and study team
Data Safety Monitoring Board established Meets monthly to review school level data and
national data to ensure participants’ education is not being jeopardized by being in the study
Participant Eligibility
18 yo or older Provide informed consent Full-time RN student Complete the nursing program with study
cohort
Ineligibility Licensed as LPN/VN or RN Accelerated RN student
The Students
Gender: 86% Female
Age: 56% 18-24
28% 25-34
16% 35+
Race: 83% White 8% Black
6% Asian 1% Native American
1% Hawaiian/PI 2% Multiracial
Ethnicity: 18% Hispanic
Student Demographics
Previous Degree: 35% (295)
62% (181) Bachelor’s Degree
41% (120) Associate Degree
3% (9) Master’s Degree
Certified Nursing Assistant: 16% Yes
Paramedic/EMT: 19
Military Experience:
28 Yes
8 Medical Corps
Study Numbers
847 Students consented and randomized411 ADN 436 BSN
54 Failed a course39 ADN 15 BSN
96 Withdrew40 ADN 56 BSN
697 Continuing on to Year 2332 ADN 365 BSN
82% rate of continuing in the study
A closer look at attrition
Clinical Instructor Demographics(N=102)
Gender: 95% Female
Race: 79% White 6% Black
7% Asian 1% Native American
Ethnicity: 9% Hispanic
Age: 20% Less than 35
24% 35-44
18% 45-54
39% 55+
Clinical Instructor Demographics
Teaching experience: 94% Yes
average 9.0 years (SD 8.7)
Current position:
43% Instructor
29% Adjunct faculty
13% Asst or Associate Professor
4% Professor
14% Other
Clinical Instructor Simulation Experience
21% No previous experience
55% Observed simulation
47% Run simulation scenarios
50% Debriefed students after simulations
38% Assessed students who have participated in simulations
23% Written simulation scenarios
33% Participated in formal simulation training
Clinical Hours
Range of total clinical hours w/o capstone:
520-776 hours
25%= 130-194 hours of sim
50%= 260-388 hours of sim
What have we learned so far…
Study Model is working Use of a trained simulation team Clear objectives 15-20 minute scenarios Having clinical instructors observe Having students observe their peers Debriefing
Skeptical faculty are now on board
Incidental Findings
Informal training/norming of clinical faculty-- more inclusion of adjunct faculty
Bonding of students in their clinical group Clinical instructors are using DML in their
post-conferences Johns Hopkins University—focus groups of
clinical instructors and how this experience is informing their teaching
Study Timeline
Current Status of Phase II
Study Students are in the senior year of their programs
Students will graduate in May 2013 We will collect NCLEX pass rates through
December 31, 2013 Final results are expected in Spring 2014
Looking Ahead to Phase III
Phase III will follow graduates of the study sites into clinical practice
Purpose of Phase III is to evaluate how well students were prepared for professional practice
Evaluations will include: Clinical competencies Safety competencies Job satisfaction Retention Stress
What can regulators take away
Who is conducting the sims? Do they have training, what kind?
What kind of debriefing? How will students spend their time while in
sim? How long are sims? Are there clear objectives for each sim? Evaluations/graded? Is the sim lab accredited?
